I am the CEO, and I was a coder in the past; now, I am not coding anymore, but still, I discuss everyday technical issues with my technical team, plan the roadmap, and design new products. I have enough technical background to vote on your REP, but if I have any doubts, I have a team of engineers helping me.
In my opinion, this REP was an attempt to promote your company products using the names of NVIDIA and other hardware acceleration providers, the TSC members, and the ROS community. You tried to standardize your approach, stating the REP was informational, but talking about standardization elsewhere, including here and on the roboperf web page:
Standards
RobotPerf aligns to robotics standards so that you don’t spend time reinventing the wheel and re-develop what already works for most. Particularly benchmarks are conducted using the Robot Operating System 2 (ROS 2) as its common baseline. RobotPerf also aligns to standardization initiatives within the ROS ecosystem related to computing performance and benchmarking such as REP 2008 (ROS 2 Hardware Acceleration Architecture and Conventions) and the REP 2014 (Benchmarking performance in ROS 2).
It has been made clear that neither the TSC members, nor NVIDIA is backing up your standardization effort, and your reaction here is a strong ad hominem attack on all of us.
Also, here you are setting yourself as the defender/representative of the community (as if the companies you are criticizing here were not part of it). The community members will speak for themselves if they like your approach, simply by using Roboperf.
Victor, while I admire your drive for change and improvement, you may need to take a breath for a minute.
I quote the specific example of your own email below.
You claim you are giving a “summary of your proposals”, yet the summary you yourself give, is nothing more than either “criticism”, or “call to action”.
I dont see a single reference along the lines of:
“Here is problem X. And here is my specific, actionable proposal to FIX problem X”.
I have been working with Victor and I have been watching his efforts on this and other REPs for a while.
My take on this - the community and the TSC would definitely benefit from more transparency.
Which means 2 practical things:
publishing full video recording of the discussions of TSC meetings
making voting non-anonymous
From where I sit it would also be interesting to understand why Acceleration Robotics was not admitted to TSC a few months back. It does not look like it was for the lack of contributions…
And I disagree once again. I’ve presented various examples above, including the fact that the TSC has increasingly been making it very difficult. That it over-ruled the WGs and went beyond alignment. Into policy creation and into contradictions. TSC changed multiple times the policies to create a WG. TSC changed their mind multiple times about WGs categorization (who gets to be official, community, etc), which as of today, I still don’t understand why this is needed (as if TSC-created WGs are of more relevance). TSC members listed and unlisted things at their discretion at About Working Groups — ROS 2 Documentation: Rolling documentation without even pinging the corresponding WG leads. There’s not even the ROS 2 Hardware Acceleration Working Group listed in there today.
I applied early this year for the ROS 2 HAWG to become a TSC Working Group (and let’s highlight there’re existing TSC WGs led by non-TSC members official members, but observers). I did so, only to get pointed to the back-door and requested to “apply to the TSC as company for an official membership”. Of course this got rejected. I got %s, but no further input. Nothing.
So no, Ingo, TSC is not primarily alignment. Aligning would’ve been recongnizing the ROS 2 HAWG work and interest as well as its numbers and contributions. What happened is very different from alignment and that’s precisely why I started this rant. Because something’s very wrong with this TSC.
I understand that you may feel frustrated and believe that I am not directly acknowledging the suggestions to advance the matter technically. I stated my disagreement with the rejection and that such reponses/alternatives are not the right path as they’ll lead to benchmarking misunderstandings. It’s my right to do so, and I blame the current TSC. This is the forum to complain and discuss it.
I recognize that emotions can run high when we are passionate about a subject (and everyone knows that I am when it comes to ROS). It is everyone’s right to feel upset, but that’s not a reason to get personal as you did. Personal attacks hinder effective communication and prevent from collecting valuable feedback that could lead to a better outcome for everyone in this community. Additionally, while I have been accused of making ad hominem criticisms, my initial comments were carefully generalized. It was only after @gbiggs rundown of TSC members requesting further clarification that I particularized my comments providing some examples directed to their positions in the TSC (not that I dislike any of them or have a personal opinion. I just don’t care). So no, there is no “making myself out to be the victim here” Ingo, I’ve been taking responsibility for each and every one of my comments, responding with clarifications. Taking responsibility for my words is the opposite of playing victim.
And that is precisely the problem Jaime that I’ve remarked above. We need TSC members that are involved into the technology, to steer the technology in the right direction. You are not involved in the technology anymore, so ideally and in my opinion you should get someone involved in the technology from your side that can help steer the project. Otherwise, you could hire tomorrow another CEO and have her/him replace you in the TSC. After all, she’d be representing eProsima.
Wrong. Acceleration Robotics is a company producing semiconductor building blocks for robots using hardware acceleration. We make faster ROS components using hardware acceleration so of course it’s important for us to have an accepted consensus to measure performance. This helps us drive our product development and services. REP-2014 was an attempt to elevate the already existing consensus in the ROS community (which is using ros2_tracing) to a standard (yes, informational, but still standardized because it already is! The whole ROS stack is instrumented as such). Trying to reflect this (again, already existing consensus in the ROS 2 codebase) in a REP was the most logical step.
And just for clarity, again, REP-2014 and RobotPerf (an implementation of the REP) initiatives aimed to establish a common ground for robotics benchmarking, not to promote any specific company. They are built vendor-neutral (which is what’s scary for some companies) and that starts with ourselves getting out there, requesting feedback for multiple years (within the HAWG) and then writing some general consensus (REP-2014) and implementing it (RobotPerf). Btw, representatives of the TSC participated in some of these meetings. So you should be aware of this (but maybe choose to ignore it).
Also, note that similar to MLPerf, RobotPerf aims to become an industry standard by adoption. Placing ROS 2 at the center of this approach encourages roboticists to use ROS for performance evaluation, benefiting the entire robotics ecosystem. Attempting to leverage ROS’ successful community standadization efforts to generate an industry standard (RobotPerf) is nothing wrong or bad for ROS, quite the opposite. The informational and non-enforceable nature of REP-2014 was intended as a guideline, not a strict rule. Its rejection by the ROS 2 TSC is a missed opportunity to set common conventions that could have positively impacted the community.
Let me give you a direct example of the relevance of a “consensus in benchmarking”: in the last RobotPerf release we disclosed that we’re observing a very concerning behavior with Fast DDS when it comes to analyzing the latency of certain perception computational graphs:
This is not new and If you were more involved in the technology, you would probably be concerned by problems like this a bit more. Having REP-2014 out there would help us all align and agree on a general way to collect metrics, benchmark and compare.
Fair enough Phil, here’s a improved version of what I conveyed before, but now including actional proposals:
TSC Issue
Comment
Suggested action
Lack of Transparency
I criticized the lack of transparency in the TSC’s decision-making process, particularly in the evaluation of REP-2014, and other WG-related matters. I called for more openness and accountability.
All decisions by the TSC should be disclosed, including voters and votes
Non-Technical Rejection
I expressed frustration that the feedback for the rejection of REP-2014 was mostly non-technical, despite the proposal feedback being discussed, reviewed, re-iterated based on initial feedback, which we addressed in detail
TSC comments and feedback should remain technical. If they don’t have the expertise, they should either defer to the corresponding WGs or invite external independent experts (without conflicts of interest) that advise accordingly
Political Nature of TSC
I criticized the TSC for being more political and less technological, and questioned the value of the current TSC setup
TSC membership shouldn’t be voted by the TSC, it should be voted by the community as a whole (open poll to all members of the community), preferrably in an open/transparent manner (again to facilitate accountability). Also, TSC memberships should be renewed and voted periodically, e.g. every year. Politics and lobbying within the TSC should be rejected.
Lack of Strong Contributors
I criticized the TSC for missing strong contributors, key players, and individuals with a strong track record in the community.
TSC should be diverse in both gender and representation. Companies (Big companies, smaller and startups) as well as community (non-affiliated membership) representatives should get similar representation counts. The overall ROS community should have avenues/methods to raise concerns against TSC members, and methods to replace them if appropriate
Anonymous Lobbying
I expressed concern about anonymous lobbying (mostly corporates, and under confidentiality) and the confidential tone of ROS TSC discussions
Confidentiality should be an exception, and never used for TSC decisions
Need for Community-Centric Governance
I called for a change in the current leadership towards a more community-centric governance model
TSC community representatives should increase to match the count of company representatives and (regardless of their affiliation) should act as non-affiliated for what the TSC matters concern
Inconsistency in Standardization
I criticized the inconsistency in the TSC’s decision to reject standardizing a benchmarking approach in ROS 2, despite existing ROS codebase consensus, despite existing accepted implementations and despite the consensus reached among those involved in its development
Accept the community standards as community standards. Don’t reject what’s obvious (e.g. adoption of ros2_tracing) for tracing and metrics collection.
Lack of Accountability
I criticized the opacity and lack of accountability in the governance structure, and called for proper mechanisms to replace inactive or non-contributing representatives
All votes should be publicly disclosed, with the corresponding voters. Voters should be allowed to provide comments to clarify their position. All TSC members should be elected by the community, accountable for their acts and/or replaced given the right circumstances
Slowing Down of Innovation
I expressed concern that recent policies and events are slowing down the innovation process in the ROS ecosystem and discouraging many from continuing to contribute
Proactive contributions aligned with ROS interests should be encouraged. Not discouraged or stopped. TSC should carefully consider the technical value of each new initiative and its implications for the ROS ecosystem
So if you don’t code anymore, you are not involved in technology. Even if you manage a group of engineers, right? Fortunately, it is just your opinion.
Enclosing others hardware in a plastic box does not make you a semiconductor company. You need roboperf to become an standard to influence the market towards buying your boxes. It is obvious. And that is why you are pursuing this so badly.
I understand you are implying here NVIDIA. Do you really think a multibillion company can be scared because of this? Perhaps you should review your position in the market.
You did this already, and we didn’t answer, neither we are answering this time. It is your policy to try to market your products using other names. You have been doing this from long time ago with Micro-ROS, Vulcanexus, Safe DDS, Fast DDS… we decided long time ago not feeding you.
We answer to many performance “contests”, and we use several performance frameworks. We periodically run APEX.ai framework, the OSRF tests to get selected as the default DDS, our own framework (part of our CI system), nav2 performance (that’s a good real example), many customer scenarios in which performance is everything, github issues… But we will not consider your framework.
Bringing in this discussion the performance of our products (offtopic), my position within the company I represent, or my technical knowledge, it is a good example of trying to deviate the conversation by ad hominem attack.
So, I reiterate: You tried to standardize your approach, subverting the process by stating in the REP was informational but talking about standardization elsewhere and using other’s names to imply consensus (NVIDIA) without their consent.
Now, as it was rejected, you attack the TSC members and its governance. I think at this point is clear to anybody why you got your REP rejected.
Jaime, while I do not wish to engage in public fights with you or a discussion about Acceleration Robotics, I feel compelled to address the attacks and misunderstandings in your response. It seems there is a fundamental misunderstanding on your part about my role and the work we do at Acceleration Robotics.
Your characterization of our work as merely “enclosing others hardware in a plastic box” (referring to ROBOTCORE RPU, an open community-oriented design we disclosed as part of our work in the HAWG[1]) is not only incorrect but also demonstrates a profound lack of understanding of the technology and the broader robotics ecosystem. Semiconductor building blocks refer to Intellectual Property (IP), or hardware designs that then turn into hardware (and or silicon, depending on whether they become soft-cores or hard-cores). Your own products run on what was once a soft-core, only to become a hard-core. The whole robotics ecosystem (and hopefully spearheaded by ROS) is going to benefit significantly from domain-specific compute architectures. That’s what we’ve been doing for a few years now. That’s what many are doing now.
Examples of some of our released products include ROBOTCORE Transform or ROBOTCORE Perception, which are ROS 2 API-compatible and give you a computing performance increase. You’re welcome to check those (or have your team look at them) and send feedback if you wish.
You’re once again misunderstanding (purposely?). I will repeat it: because “you are not involved in the technology anymore, you should get someone involved in the technology from your side that can help steer the project (participate in the ROS 2 TSC)”. I’m not making a statement about you personally, or about technology in general. I’m questioning your capabilities as a ROS 2 TSC representative and how much that brings to the community.
Your decision to dismiss our framework outright without consideration is disappointing, but not surprising given your apparent lack of understanding of our intentions and the work we do. Accusing me over an over of ad hominem attacks and attempting to deviate the conversation is a diversionary tactic that does not address the substantive issues at hand.
Your attack on my position, and our products is very unprofessional. As a TSC representative, it is your responsibility to represent the interests of the ROS community, not just your own or your person or your company’s. Your response, however, seems more focused on attacking me and my company rather than addressing the substantive issues at hand. Again, asking for more accountability from TSC members.
Your comments about NVIDIA and your accusation of us using other names to market our products are ridiculous, baseless and unfounded. We have always been transparent about our collaborations and references to other projects and companies are made in the appropriate context.
This (discussion) is about the ROS community, not the overall market. I’ve been extremely critical myself towards the attitude of companies like NVIDIA over multiple threads here in Discourse, calling for technology alignment (as opposed of forks and re-marketing), calling against their lock-in approach by forking and changing APIs, calling for community-aligned benchmarking (using what’s already a standard, but not qualified to become a REP, that is ros2_tracing) and also complaining about those times wherein NVIDIA forked packages and changed their license to make it less commercially friendly (packages many of us here have contributed for a decade).
This is one of those things TSC members like you should be looking at (and btw, as acknowledged above, we got positive “reactions” out of these past discussions).
And again, I reject your view. It’s not my approach, it’s the community approach. Not because I say so, because it’s been embraced for years now. The whole ROS 2 stack is instrumented with ros2_tracing which is the basis of REP-2014 (again, something you’d know if you were closer to technology), which makes me confident that it should get standardized, and which triggered this whole rant. There’re even DDS implementations that have been instrumented with ros2_tracing and/or LTTng (e.g. CycloneDDS) giving robotic engineers the capability to better understand their dataflows. It’s important for you to understand that this is not me being right, it’s what’s out there, what’s adopted. When/if your group @Jaime_Martin_Losa decides to align to the trend (and instrument your DDS solutions to align with LTTng/ros2_tracing), you’ll be aligning with the community, not “feeding me”.
By the way Jaime, you’ve repeatedly pointed out that there’s no consensus. But there is, even with NVIDIA. As I explained above, the rejected the use of ros2_tracing (grey-box performance benchmarking) and instead decided to re-implemented ros2_benchmark (black-box performance benchmarking). We studied their approach, forked it to improve it and integrated it in both REP-2014 and RobotPerf (empowering you to do both). So yes Jaime, we’ve worked towards consensus from a technology point of view. And I think we have it.
From your words, It is clear that the rejection of the REP has more to do with politics and personal biases than the merits of the proposal. Your responses @Jaime_Martin_Losa only reinforces this perception. It is disappointing to see this level of unprofessionalism from a TSC representative. Instead of attacking those who are trying to contribute to the community, perhaps it would be more productive to focus on constructive feedback and working towards solutions that benefit the entire community.
Repeating myself, I recognize that emotions can run high when we are passionate about a subject. I’ll try to be more constructive re-iterating those proposals I believe are necessary for situations like this to be avoided in the future (small edits included on each proposal stated before, hopefully improving them):
ROS 2 TSC Issue
Comment
Suggested action
Lack of Transparency
I criticized the lack of transparency in the TSC’s decision-making process, particularly in the evaluation of REP-2014, and other WG-related matters. I called for more openness and accountability.
All decisions by the TSC should be disclosed, including voters and votes
Non-Technical Rejection
I expressed frustration that the feedback for the rejection of REP-2014 was mostly non-technical, despite the proposal feedback being discussed, reviewed, re-iterated based on initial feedback, which we addressed in detail
TSC comments and feedback should remain technical. If they don’t have the expertise, they should either defer to the corresponding WGs or invite external independent experts (without conflicts of interest) that advise accordingly
Political Nature of TSC
I criticized the TSC for being more political and less technological, and questioned the value of the current TSC setup
TSC membership shouldn’t be voted by the TSC, it should be voted by the community as a whole (open poll to all members of the community), preferrably in an open/transparent manner (again to facilitate accountability). Also, TSC memberships should be renewed and voted periodically, e.g. every year. Politics and lobbying within the TSC should be rejected.
Lack of Strong Contributors
I criticized the TSC for missing strong contributors, key players, and individuals with a strong track record in the community.
TSC should be diverse in both gender and representation. Companies (Big companies, smaller and startups) as well as community (non-affiliated membership) representatives should get similar representation counts. The overall ROS community should have avenues/methods to raise concerns against TSC members, and methods to replace them if appropriate
Anonymous Lobbying
I expressed concern about anonymous lobbying (mostly corporates, and under confidentiality) and the confidential tone of ROS TSC discussions
Confidentiality should not be part of the TSC on a general basis and never used for TSC decisions, or to justify the TSC. Members can establish confidentiality outside of the TSC if needed. Nobody should be put in the position of “hey, resign because you’re breaking confidentiality”. Transparency should be an inherent part of what the TSC does.
Need for Community-Centric Governance
I called for a change in the current leadership towards a more community-centric governance model
TSC community representatives should increase to match the count of company representatives and (regardless of their affiliation) should act as non-affiliated for what the TSC matters concern
Inconsistency in Standardization
I criticized the inconsistency in the TSC’s decision to reject standardizing a benchmarking approach in ROS 2, despite existing ROS codebase consensus, despite existing accepted implementations and despite the consensus reached among those involved in its development
Accept the community standards as community standards. Don’t reject what’s obvious (e.g. adoption of ros2_tracing for tracing and metrics collection, when the whole ROS 2 stack is already instrumented accordingly).
Lack of Accountability
I criticized the opacity and lack of accountability in the governance structure, and called for proper mechanisms to replace inactive or non-contributing representatives
Better accountability by recording meetings. Also, all votes should be publicly disclosed, with the corresponding voters. Voters should be allowed to provide comments to clarify their position. All TSC members should be elected by the community, accountable for their acts and/or replaced given the right circumstances
Slowing Down of Innovation
I expressed concern that recent policies and events are slowing down the innovation process in the ROS ecosystem and discouraging many from continuing to contribute
Proactive contributions aligned with ROS interests should be encouraged. Not discouraged or stopped due to lobbying and market interests. TSC should carefully consider the technical value of each new initiative and its implications for the ROS ecosystem. If in conflict, other companies’ vision about such new contributions should be discounted
Inequitable Working Group Establishment
I criticized the current exclusion and classification among Working Groups (WGs), as we are all part of the same community and represent the same interests, whether on the TSC or not.
Establish more homogeneous WGs that do not exclude or classify members based on their affiliation with the TSC or any other organization. All members of the ROS community should have equal opportunities to participate and contribute to the WGs.
Again, you’d know this if you were technically competent and or would’ve been tuned to the Hardware Acceleration working group activities. ↩︎
After discussing this within the moderator team, we’ve decided to close this thread.
We think it is important to have open and honest discussions about how the ROS project is governed, and be responsive to the wishes of the entire community. However, this particular discussion has degraded into ad hominem attacks, hear-say, and last-wordism that is anathema to a productive conversation. Litigating past TSC discussions isn’t going to get us moving in a positive direction. Positive discussions propose better approaches on a go-forward basis. Moreover, we’re not going to retroactively change any past TSC decisions by fiat in a Discourse thread. Finally, and most importantly, discussions about changes to project governance need to happen in a neutral and dispassionate manner; the quieter members of the community are not going to weigh in during a shouting match.
From where I am sitting, chairing the TSC, I am always happy to clarify the discussions that have occurred, and I would also ask that there be an assumption of good faith with respect to TSC operations. Finally, if this is the type of work that interests you, TSC Community Representative elections are also going to kick off the week after next, and everyone is welcome to apply.
For the time being I am going to ask everyone to take a break and find more constructive means of addressing project governance.
and having brought hundreds (both ROSers and externals) of developers together, following what’s established at 