@JRTG Thank you for the feedback. If it was so obvious that it was bound to be rejected due to issues you mentioned, shouldn’t these have been brought in the public discussion. We have always been open to criticism and feedback and thus the rep has gone through multiple iterations since first draft is September 2022. There have been various contributors/commentors on the REP and they have been respected with their feedback and changes made. The community thus had a consesus with the REP and its value. Very few TSC members were actually gave their feedback on the REP(which again was taken into account) but most did not. The rejection reasoning is not clear enough and an open discussion with the TSC to account for their feedback and transparency is what we are asking for!
Adding on to @PratNag’s point, there’s been a pretty strong consensus in the Hardware Acceleration Working Group about this with participation even from big silicon vendors. Will request and highly encourage to participate in the HAWG meetings to see the same!
@tomoyafujita, thank you for your question and for being open and neutral. To answer your question, my reasoning is mostly based on 1) past experiences with/in the TSC (you may remember I was one of the early members of the TSC), 2) plenty of discusions with companies operating in the ROS space whose contributions didn’t move forward or got rejected while trying to participate in the TSC more actively (and with contributions in mind) and 3) digesting the information publicly available from the TSC Meeting Minutes for the last few years (I don’t claim to have read everything, but a pretty decent amount to generalize). This information (3)) lacks details and transparency for the community (which a few get to represent and discuss monthly in the TSC, sure, but still, the TSC is the ROS project/community’s TSC and each one of these representatives should act responsibly to it). Again, there should be accountability. I think demanding full transparency with regard relevant votes/decisions is very fair.
As I was hinting above, many current TSC members have not authored a REP or been exposed to or ruled by the policies they are setting, which have changed rapidly in recent years. Due to this policies, despite ROS being an open project, it has become increasingly difficult to contribute, especially with new ideas, unless you are in the TSC or “own” the piece you aim to contribute. We need a more diverse TSC committee that encourages and includes more community representatives, at least as many as corporate ones, and who actively engage with the ROS community’s issues and needs. If representatives do not pursue the community’s interests with evidence or are inactive, there should be proper mechanisms to replace them.
Let me give you a practical example: REP 2007 started sponsored by a TSC member (great contribution, by the way) on Jan 28, 2021 (PR) and it took 4 months to merge it. REP-2008 started on August 25, 2021. After much (much!) insisting, various TSC presentations, TSC discussions and TSC reviews, it was pushed back to the ROS 2 Hardware Acceleration Working Group (who was the original proponent) and approved in 16 months (as a “Draft”). I’m happy this went through but it was a terribly painful process, which involved explaining things over and over to TSC members who were not technically qualified on the topic. Before REP-2008, no REP proposal went through such scrutiny. I have more examples if needed.
In recent years, the TSC has overruled, especially with new policies that were not publicly discussed or accepted but introduced on-the-go. These policies and recent events are slowing down the innovation process in the ROS ecosystem and discouraging many from continuing to contribute.
Consensus was reached among those that decided to get involved in this work. This REP has been discussed throughout those meetings for a very long time (since Sept 29, 2022). Many of these meetings were very well attended 80+ participants and included dozens of participants from the ROS community, including experienced deverlopers and newcomers. This REP also encouraged repeatedly the participation of silicon vendors (who are generally very interested in benchmarking) and received reviews from various of them. We particularly insisted on NVIDIA to join/support the effort, but instead, they decided to go independently and disclosed ros2_benchmark instead of aligning and joining efforts.
Despite NVIDIA not collaborating, in an attempt to be more inclusive (and following from prior TSC input), we went ahead and within RobotPerf (which is a reference implementation of REP-2014) included ros2_benchmark as one of the options, supporting NVIDIA’s take. We’ll be publishing an article about this shortly.
Accordingly, I don’t accept the inconsistency you remark. We’ve tried really hard to reach consensus, even when ignored.
Open to criticism and feedback != alignment and consensus.
This has not been our experience. Much of our feedback has been attacked or ignored.
Our monitoring of HAWG has been used to imply, as it is in this thread collaboration and agreement. This misrepresents a lack of alignment and consensus hence we stopped attending HAWG.
Our work on bringing accelerated computing to ROS 2 requires we regularly measure performance. We could not do that with the requirements as stated in the rejected REP. We released tooling for ROS 2 benchmarking in open source and ROSBag datasets in Creative Commons v4 Commercial to provide benefits of the tooling and choice to the community.
Perhaps you are overlooking your victory . Driven by your initiative, a commercial quality tooling for benchmarking accelerated computing in ROS 2 is available in open source.
@vmayoral First, to be clear: I have no personal interest in benchmarking and I do not have an opinion whatsoever about which approach to ROS 2 benchmarking is better or preferrable or whatever.
That being said:
You claim e.g. “consensus is reached” and “Most aspects in the feedback were discussed and addressed”.
Either the REP is intended to standardize the benchmarking approach for ROS 2, in which case:
It should not be an informational REP, and
At the very least it should address these and these comments, which it does not.
Or, the REP is intended as an informational REP, but
It is not at all written in an informational tone,
Not even this small request was applied to clarify the informational status.
I take note of your ongoing quarrel with Nvidia, and I take note of your complaints about the TSC procedures. Maybe you have a point, maybe you don’t. I am not voicing an opinion on that, I am voicing an opinion on this REP proposal. This proposal:
Alignment and consensus with whom exactly? That is where our question lies. You raised your voice and were not happy that some of your asks were not incorporated and thus it is understandable for you to not be happy with REP. But the REP is being voted by TSC most of whom have not been a part of the process. Of Course all community REPs should not be accepted and there should be a due process for quality control, but the current procedure to get a REP passed is just to make sure that all TSC members are appeased which for a community driven ecosystem does not make sense.
I have huge respect for most of the TSC members and organisations and there needs to be industry involvement in this process but shouldn’t the community get an equal say as well? Apart from REP 2014 which we initiated and got through after a lot of difficulties, none of the REPs accepted to my knowledge have been community driven. We are not even given a chance to put forward and have an open discussion with TSC as we are not a part of it, thankfully @fmrico has been supportive and brought forward our REP to the TSC.
We would have been more open to address the suggestions if there were more active contributions rather than just comments. Lack of resources on our end makes it difficult to do incorporate all requests.
The comments that you mentioned if you continue reading you can see Steve replying back accepting the position of the REP as informational.
This was immediately replied to here mentioning that it is very clearly mentioned at the start itself that this is an informational REP. See REP 0101 for example and other informational reps. We are just following the standard ways on how REPs are written. None of these REPs were scrutized or asked to make it make it explicitly clear to everyone over and over again by mentionining in 10 different more places that this is an Informational REP.
You are still bound to have your opinion to reject it and I respect that but downplaying that we have not accounted to proposed changes whatsoever and us being critical of TSC and it’s processes is what I don’t agree with.
Anyways I don’t think we should put more of our mutual energy over this. I am sure things will change and something better comes out of this in the future! In the end we all want the ROS Community to succeed!
I definitely acknowledge the value of this work, full stop. However I’ve expressed in the PR reviews I’m not sure an REP is the right place for this work in no uncertain terms. Honestly I think an academic paper with a more user-friendly blog post describing this method / process as a reference is the right solution. An REP could then follow after gaining sufficient traction and refinement over years until its reached effective consensus.
While the informational REP does meet the intent of this work (e.g. provide information) that doesn’t mean that that information doesn’t need consensus across a broad spectrum of the community or that because it is submitted as an informational REP that that is really the right type of REP for the class of work described. I think it needs to better fit in either camp:
Detail to be totally prescriptive as would be required of a standard covering all major needs (and be proposed as a standard)
Consolidation of opinions and general consensus across the more broad community than a single corner for informational purposes around a software package so it can omit the prescriptive details – over years of usage an refinement. If its not the “defacto standard” or “there is no genuine alternative”, its not there yet.
To a degree, the continual pushing is what makes it somewhat a problem and creates the tension. You also have to convince folks of your ideas, not just address the specific technical concerns that are articulated as action items. Then once running out of action items, demanding a vote to ratify.
Baking time is not uncommon in these kinds of documents and delays on the order of year(s) should be expected to give enough time for consensus to be reached / fully appreciated. I have REPs open for > 1 year and I’m pretty certain they’ll get there eventually, but baking time is part of the process. Every ~6 months when I kick the tire, I can feel the gears moving my direction and new thoughts brought up which make it better. I think this is an important point to make. The gears of standardization turn slowly and its hard to speed up “this method is the best method” until you have years of use under the community’s belt.
Just because there isn’t consensus today, doesn’t mean that there never will be.
I think this is perhaps where we lose each other. That’s certainly a concentration of important folks, but not the exclusive set of them.
On the TSC
Sitting in those meetings, I don’t think that’s really true. I think there’s less and less of everything as we wait for whatever the new system that is being put together by OSRF to be put together
I can hardly look at the list of TSC members on the attendees list above and note someone that’s not contributed to ROS with relevance.
I reject in entirety your characterization of the TSC operations. I won’t say the TSC doesn’t have other issues, but politics and confidential lobbying isn’t one of them (currently). And certainly no one that I’m aware of is politicking or lobbying about REPs now, in the past, nor expected in the future. It won’t surprise anyone that is a more hot topic when default DDS vendors are being discussed, but little else. We’re not an evil cabal in smoky back rooms confidentially gate keeping the ROS community. Frankly most meetings little else happens more than getting a run-down on everything that’s going on and @Katherine_Scott giving a dirty look to all the companies not filling in their contribution reports.
That’s absolutely ridiculous. If that were true, I’d be a TSC member and not an observer there to present the Navigation Working Group summaries. There are a handful more comments like this that are outlandish but I don’t think merit further asks for clarification, because I’m sure there is none to give rooted in reality.
This seems to me just a general airing of perceived grievances not substantively backed by the reality of things. If you have specific and productive policy guidelines that you’d like to propose, I think there’s a body of folks that are open to hearing about it. But if you just want to complain and attack, that’s not going to change anything (I know, I love complaining ).
This is going to be a long post, because Victor has made a number of points that I want to address in turn for the benefit of the whole community.
On the feedback from the TSC
TSC members who felt that their feedback was relevant to improving the proposed REP document itself provided commentary on the PR. Other TSC members may have felt that their informed opinions were less useful as feedback and more useful for allowing all TSC members to make a fully-informed vote. These opinions don’t necessarily need to be conveyed to the authors of a proposed REP nor made public. TSC members are not required to comment on or contribute to a proposed REP in order to take part in the review process.
Furthermore, the feedback given by the TSC as part of the voting process is an ad-hoc process change made by @Katherine_Scott, the TSC chair. This change was made because it is important to have more transparency on why TSC members voted the way they did, allowing REP authors to potentially address those concerns and resubmit the proposed REP.
The feedback provided after the vote by the TSC is in agreement that the TSC would prefer the content you are proposing to be in the documentation rather than as a REP. This is probably in part based on the claim in your proposed REP that, although you state that the proposed REP is informational, the “guidelines in here are intended to be a living document that can be updated as new information becomes available.” REPs are explicitly not meant to be living documents. They are set in stone once they are accepted and published, and changing a REP means making a new REP and deprecating the old one. This is necessarily a long process. On the other hand, updating a documentation page is an easy and common process involving a simple PR.
On informational REPs
Although the original REP describing the processes around the REP (written in 2010, since when a considerable amount has changed in our community) does include the informational REP concept, this is not a requirement for proposed informational REPs to be accepted, particularly if there may be a better way to present the information.
Moreover, informational REPs are still meant to contain information that is broadly agreed upon. Reading over the PR discussion and relevant threads on Discourse does not convince me that there is a broad agreement, let alone a consensus. Perhaps the TSC members felt the same.
I also note that you have mixed up “informative” and “standardised” in multiple places in your posts. You state that the REP is intended to be informative, then make statements such as “TSC voted 3/12 against standardizing a benchmarking approach in ROS 2.” The REP cannot be both informative and set a standard at the same time. If your goal is truly to get the information out there so as to inform users of ROS one way to do benchmarking in order to encourage more of it to be performed, then a page in the documentation would be sufficient. An informative REP shouldn’t hold significantly more value for you, especially given its lower visibility.
On the REP process
I agree that the REP process needs an overhaul. Fortunately the OSRF is working on a new REP process as part of its year-or-more-long transition away from the old BDFL project governance model that the OSRC represented. We will call for community input into this at the appropriate time.
However, we can’t revise the process and apply it retroactively.
The current process, however ad-hoc, is the process we have at this time and we must follow it until a better process is put in place. Part of this process is a review by the TSC of any proposed REPs; this review does not require that any TSC members be involved in drafting or commenting on the REP itself, and not doing so does not mean that the TSC has not been a part of the process. Neither do TSC members need to be “technically qualified on the topic” or experts. Being able to explain a REP to non-experts is part of the point of the review.
And finally, no one is required to make “active contributions” to a proposed REP to have their comments heard or acted on by the authors. If a proposed REP does not incorporate all requests simply due to available time, then that proposed REP should not yet be accepted.
On TSC membership and its expertise
You seem very unhappy with the way that new members join the TSC. The requirements for a new member joining are open and published in the TSC Charter and in the Project Governance section of the documentation. There are no TSC members who “enter[ed] by the back-door” because there is no “back-door”. The only members who did not go through the current entry process are those founding members who were part of the initial batch of membership or joined shortly thereafter, before a more formal process for joining involving REP 2005 and an incubation period was put in place.
I’m not aware of “10+ startups” who have applied and been rejected stealthily. All membership applications that I have seen come to the TSC membership have involved a prepared document from the applicant, discussion by the TSC (confidential, so opinions may be honestly expressed), and an anonymous vote. The applications are documented in the public minutes. In the past couple of years I can think of one company that was rejected for not meeting the requirements, and one company that went through the same process and was accepted. I do know that one company has been removed recently by a decision of the TSC for lack of attendance, and one other company was at risk of being removed until they clarified their standing.
I also want to make it clear that TSC membership is conferred on organisations, who assign someone from their organisation to represent them on the TSC. Organisations may change their representatives at any time. This happens for a variety of reasons, including internal promotions, leaves of absence, change in role or responsibilities, and resignation.
I wish to also ask you to scale back the ad hominem attacks you are making on the TSC members. Using descriptive phrases like “the corporates” and “backed by X company”, and stating that most of the members of the TSC have “never contributed to a REP, or ROS with relevance” are not helpful, not correct, and are divisive.
Running down the list of the TSC representatives who attended the June TSC meeting (I selected June because July and August meetings typically have lower attendance due to summer vacations):
Ralph Lange and Christian Henkel have been involved in ROS activities for years, and lead the groups in Bosch that do make heavy contributions.
Jaime Martin Losa’s company produces the FastDDS package on which much of ROS relies.
Angelo Corsaro’s company produces the Cyclone DDS package on which much of ROS relies.
Chris Lalancette from Intrinsic is the project leader for ROS.
Lou Amadio from Microsoft is an engineering leader who has shepherded many important contributions to ROS from his company.
Tomoya Fujita from Sony has contributed more to the core of ROS than most of us, and other engineers from his company are likewise significant contributors.
Ian McMahon from TRI is a very long-term contributor and represents a heavy user of and contributor to ROS.
Henning Kayser and PickNik have been involved in ROS for many, many years and made huge contributions.
Jeremie Deray and other engineers from Canonical have contributed a significant portion of the security capabilities of ROS. (The Canonical representative has changed to Mirko Ferrati since the June meeting was held.)
Michael Orlov from Apex.AI leads the development of ROSBag2 and other engineers from Apex.AI have been heavy contributors for a number of years.
Alberto Soragna and other engineers at iRobot are heavy contributors and have provided much of the current knowledge on executors.
Adrian Macnell from Foxglove (a startup) and his engineers (although he’s one, too, but now he has to run a company) provide arguably the best visualisation solution for ROS, contributed the MCAP format and its myriad of client libraries, and have made heavy contributions to ROS.
Jerry Towler from DEVCOM is a long-term contributor to ROS.
Aaron Blasdel from Amazon has a long history of contributing himself, as do many other engineers from Amazon.
Matt Robinson is the current lead of ROS-Industrial.
Andrei Kholodnyi and Wind River are some of our leading expert contributors on real time execution.
Although some of the people on that list are now in management positions, I’m not aware of any who are not engineers. I certainly wouldn’t describe any of them as “corporates”.
I think it is also worth explicitly noting that while the majority of the TSC members are “from industry”, that does not make them a separate group from “the community”. The ROS community is made up of a diverse mix of hobbyists, students, and professionals. Giving “the community” an “equal say” to “industry” implies that those members who use ROS for work are somehow separate from the rest of the ROS users. Although TSC members who represent a company are in part representing their company’s point of view, they all do it in a way that is balanced with their views as part of the community and what is best for ROS as a whole.
For a purely non-industry point of view we have the three community representatives. I have seen complaints that some of the community representatives are not active. If you do not agree with the activities of a community representative, the correct course of action is to not vote for them the next time they stand for election. This is one reason why I encourage everyone to think seriously about who should be their community representatives and vote accordingly. The next community representatives election will be starting in September.
Politics and “anonymous lobbying” in the TSC
I’m not aware of any politics or “anonymous lobbying” in the TSC. If you are, I encourage you to provide specific examples so myself and Kat can correct that behaviour.
I also encourage you to clarify your statement that working with the TSC is “complicated”. Broad, generic statements like that do not help us improve processes. Specific problems and examples of them do.
On the ability to contribute to ROS
Can you please be more specific about what policies the TSC is setting that have “changed rapidly in recent years”? I’m not aware of a rapid change in how ROS is run as a project other than the instantiation of the TSC itself.
I’d also like to hear specific examples of it being difficult to contribute to ROS, so we can understand why someone found it difficult to contribute, and try to improve that experience. However I note that the TSC does not gate-keep contributions to ROS other than the REP process, so there shouldn’t be any involvement by the TSC in making new feature or bug-fix contributions to the source code.
On confidentiality and transparency in TSC discussions
The TSC has rules about confidentiality for good reasons. Members must feel free to give their opinions honestly and openly without fear of reprisal in any way in order to encourage free discussion. Confidentiality is rare in the TSC, and typically relates to things such as forthcoming product launches, interpersonal discussions, and potentially contentious votes.
Confidentiality may seem to contradict the desire for transparency, but the TSC has so far been able to balance both. We think the trade-off of allowing confidential discussion to occur when necessary is worth it.
The TSC charter, which you linked to, is clear that any information not given in the public minutes should be considered confidential by all TSC members. It also specifically states:
1 - Statements made by individuals in discussions surrounding a vote or topic of contention which are not reflected in the publicly posted minutes should be considered confidential.
2 - Statements made by individuals regarding companies, research groups, individuals, or other entities for which they are not also members should be considered confidential.
[…]
5 - Affiliates of TSC member organizations or guests of TSC events privy to confidential information will be held to the same standards set out in this section.
If a member of the TSC has been passing on to you confidential discussions and comments from other TSC members, then that TSC member is violating the TSC charter. If the member does not agree with the rules of the TSC, then they should either raise this during a TSC meeting (and comply with the charter until it is changed) or voluntarily leave the TSC. Breaking the rules just because they do not agree with them is not acceptable or professional conduct.
On the “slowdown” in Discourse engagement/involvement
We’re not seeing a slowdown in Discourse engagement and involvement. At worst, it’s stable. On some statistics, it’s growing. This can be seen in the Annual ROS Metrics Report that @Katherine_Scott spends a great deal of time and effort putting together for the community. It is also visible in the inbuilt Discourse statistics, shown below. The following data is monthly; it is not cumulative.
Similarly, a long post responding to the various points above:
See below. Answered with @gbiggs points in more detail. Shortly, most input we got (again, see here) of REP-2014 concerned opinions about what should be in such a REP (and/or how it should be written). You seem to be into writing papers a lot these days so you’ve probably noticed you’ll encounter lots of types of reviewers, some more constructive, some more qualified in the art, but the least you’d expect from all of them is to have writen at least one paper. Otherwise, It’d be infuriating to have them critize the structure of your paper and its content, since they’ve never written one.
You’re implying here that I’m just complaining and pointing fingers. That’s just wrong. See previous comments. If what you need is a summary of my proposals (besides the complaint concerning the rejection of REP-2014, that I believe I’ve made clear) here’s one for you:
Lack of Transparency: I criticized the lack of transparency in the TSC’s decision-making process, particularly in the evaluation of REP-2014, and called for more openness and accountability.
Non-Technical Rejection: I expressed frustration that the feedback for the rejection of REP-2014 was mostly non-technical, despite the proposal feedback being discussed, reviewed, re-iterated based on initial feedback, which we addressed in detail.
Political Nature of TSC: I criticized the TSC for being more political and less technological, and questioned the value of the current TSC setup.
Lack of Strong Contributors: I criticized the TSC for missing strong contributors, key players, and individuals with a strong track record in the community.
Anonymous Lobbying: I expressed concern about anonymous lobbying (mostly corporates, and under confidentiality) and the confidential tone of ROS TSC discussions.
Need for Community-Centric Governance: I called for a change in the current leadership towards a more community-centric governance model.
Inconsistency in Standardization: I criticized the inconsistency in the TSC’s decision to reject standardizing a benchmarking approach in ROS 2, despite existing implementations and the consensus reached among those involved in its development.
Lack of Accountability: I criticized the opacity and lack of accountability in the governance structure, and called for proper mechanisms to replace inactive or non-contributing representatives.
Slowing Down of Innovation: I expressed concern that recent policies and events are slowing down the innovation process in the ROS ecosystem and discouraging many from continuing to contribute.
Good for @Katherine_Scott and I acknowledged this piece already, as for the rest, I disagree. We got no such “feedback with the possibilities of addressing those concerns and re-submit the REP”. The REP has been Rejected and marked as such and as per my understanding of the REP process, there’s no way out now. The message hasn’t been, Víctor and group, please address these concerns and re-submit, the message conveyed as I understand it has been Víctor and group, this REP has no room as REP, so we’ll reject it with this feedback. We encourage you to make a docs page.
We both know in both cases (REPs or docs) can be changed with a simple PR, let’s not over-complicate.
This point has been made very clear. We’ll try and follow what’s being recommended here.
That’s clearly not what happened in here. This is brought out of context. REP-2014 was discussed before the summer and rejected over the summer. It’s unreasonable to expect any prompt reactions to comments dropped in the PR over summer time when it took months and multiple presentations only to get it in the agenda. If any of those comments were of relevance for the decision, the such a decision should’ve postponed with feedback back to the authors.
If what you refer to is my complaint against NVIDIA’s attitude of fork-and-use(but-not-contribute-except-for-marketing) then sure, you’re right and I mean it. We had a similar situation in REP-2008 reviews. TSC requested that we had NVIDIA accepting it as well (phrased as “buy-in from the larger acceleration community”). NVIDIA ignored this whole thing completely (difficulting interoperability across accelerators). The TSC insisted and I had to go ahead and implement it just for the sake of addressing that red line established at the time. The TSC and REPs efforts should favor those willing to invest time on topics of interest, not impose hurdles because one company is not aligning.
Answering this in details would take lots of time. Lots. I’ll try to be short:
There’s plenty of organizations whose applications took many months while for others it happened overnight jumping over the TSC intake process. Not implying these organizations should (or should not) be there, but can you confirm Intrinsic went through the scrutinity of the whole TSC intake process (and note I was present when the hand-raised vote happened for Intrinsic)? Can you confirm Foxglove went through it as well and had all the credentials generally requested concerning REP-2005? Can you confirm (other than the founding members, as you call them) that all the rest went through it?
You also imply you don’t know of various startups applying and being rejected yet I myself did so, multiple times, with multiple companies, over multiple years obtaining creative input. Also discussed this with other frustrated applicants. Not only the information you have seems incomplete, but you point to a link which contains extremely few details about this. Where’s the transparency here? Where’s the associated information with each application? Is this also “confidential” so that the TSC members can express their views without fear?
From an applicant’s perspective the TSC mostly seems an invite-only-club. Not something you can generally apply to.
Feedback is meant constructively and intends to convey the message that most TSC represent big companies and act aligned with that (even at the point of throwing personal attacks). Also, you’re brought this out of context. My exact words were “We need a more diverse TSC committee that encourages and includes more community representatives, at least as many as corporate ones, and who actively engage with the ROS community’s issues and needs”. I mean corporates from the perspective of individuals who run company’s daily operations (and that I believe is its meaning), but not as engaged with the technology.
Don’t get me wrong, I know very well the relevance of those big names/companies and how we all fund the efforts we then invest into ROS but that doesn’t mean I can’t have my own opinion on how things could work better from a community governance perspective and that is precisely what’s being expressed here.
A better TSC structure would have more startups/smaller companies (which unsurprisingly and acknowledged by most, are the ones who contribute the most when they have a TSC sit) and more community representatives. At least as much as “corporates” . Again, not meant negatively. It is what it is, and in my opinion, not what we need the most right now. We need to incentivate contributions strongly as the transition from Intrinsic is happening.
Again, the majority of the inviduals representing these organizations have not participated in a REP, nor are known for their outstanding ROS contributions. You discard these arguments but I ask you again to reconsider and re-evaluate things. Let me touch on a few persons in that list that I’m happy to voice my opinion about. Taken from the official source of TSC representatives https://docs.ros.org/en/rolling/The-ROS2-Project/Governance.html:
Christian Henkel (Bosch, let’s not mix things up, if Ralph’s not entitled anymore he should be an observer at best and should have no vote): Don’t know Christian, AFAIK he has not partcipated in REPs. I also don’t know of many contributions from his side neither.
Jaime Martin Losa (eProsima): Jaime is a good example of a corporate with no relevant ROS contributions
Mirko Ferrati: don’t know of any contributions that justify him being here, neither involvement in REPs. By the way, what you claim about Canonical is incorrect. They have been rotating the TSC members a few rounds in the last few years and have not contributed a significant portion of the security capabilities of ROS. In fact, many of the security issues in ROS 2 remain as of today, many years after reported. As I said before, there’s nobody taking ownership of the real security issues in ROS 2. Without meaning to criticize the work of Jeremie and other Canonical colleagues at the SWG (I know they are pressed by other matters and have very limited resources) contributions security-wise from Canonical have mostly been small utilities and a great marketing use of ROS.
Patrick Musau: was selected but didn’t show commitment with the project. Various others in the past requested an avenue to replace him. These requests were answered with a “please vote someone else next time”. Hilarious.
Oliver Michel: I saw very few contributions from Oliver and most concerning his own products and campaigns. I don’t know enough to have a strong opinion, so I’ll jut make this observation: It’d be great if TSC members (regardless of whether they are community reps, corporates, startupers, etc.) are encouraged to contribute to those aspects of most relevance for ROS, besides their own products. TSC members are requested a portion of a PM to participate. Let’s make good use of that
I’ll repeat my words above the current setup of TSC “thought-leaders” (most of which have never contributed to a REP, or ROS with relevance) adds very little value.
Let me explain my thoughts differently: companies nominating TSC representatives should ensure such representatives have a strong ROS track of records by themselves and as contributors (in the various forms that could take, but existing). Why? Because 1) we want direct encouragement for companies to hire ROS robotics engineers (that’s a great reason to participate in this community and encourage people to contribute!), 2) it’s crucial they understand the technical impact of their decisions and repercussions across the codebase, and 3) those TSC members themselves will contribute directly and remain technical, not political.
For example, Sony (Tomoya) is a great example. He’s (as far as I know) one of those who’s walking miles beyond his solutions/software/products to improve the overall ecosystem. He’s not only sending PRs across the ROS stack, he’s reviewing REPs and from what I’m seeing here/life, also trying to constructively collect feedback. We need more Tomoyas and less managers in that list (since corporates seems rejected).
I’ll paste in here again my comment above implying that the overuse of confidentiality together with the current TSC which is mostly represented by big companies (and corporates) is leading to “confidencial lobbying” or “anonymous lobbying”
Please see:
Contributing to ROS is not just sending a PR (as we’ve all acknowledged repeatedly). It’s also about coordinating a subgroup of the community, pushing forward a new consensus to avoid reinventing the wheel (aka REP), organizing meetings, etc.
I have experienced plenty of difficulties with the ROS 2 Hardware Acceleration Working Group. Soon after running the group for a while successfully we got changed into a “community” category, followed by changes in calendars and events, followed by its complete removal from the official list of ROS 2, followed by removal of (access) rights to events we had previously created, etc. This has been a pain. Received plenty of complaints about it from members of the group and they were right. It became hard to track things after so many changes. You claim this is not TSCs domain but I recall reading in the TSC notes about some of these changes so that’s why I made that assumption. Whoever’s responsible for this, it’d be great to take it into account.
I’ll be brief in here and just say that I don’t agree with the argumentation nor think confidentiality is rare based on my own experiences. I believe TSC members should be transparent with their own votes and opinions, and help community members achieve their goals, not present obstacles. I believe the current setup favours lobbying and I’d encourage OSRF to consider recording TSC meetings and disclosing them.
Good thing then! You own the data @gbiggs and I don’t believe to have access to the information you point to. If you say so, I can believe it. Happy to be wrong here. The following is what I’ve had in mind for the last year approximately and what I’ve based myself on to make my statements:
I’m merely stating the fact that while you make certain claims about people from the TSC saying certain things to you privately, these claims are not supported by anything besides your claim. I respect that you may want to protect your sources, but if this is to be taken seriously, we need to hear it from more than just one person.
That is your view. I stand by my position that everything that is really relevant is discussed in the working groups in an open and transparent fashion and that the TSC is primarily alignment, and I can only re-iterate that the only datapoint you’ve brought so far is the decision regarding the REP proposal.
You’re claiming that I do this in bad faith. Have you considered that I might be doing this because I’m fed up with one man making himself out to be the victim here, and being blind to the valid criticisms of his proposal and genuine attempts to bring the matter forward on a technical level.
I am the CEO, and I was a coder in the past; now, I am not coding anymore, but still, I discuss everyday technical issues with my technical team, plan the roadmap, and design new products. I have enough technical background to vote on your REP, but if I have any doubts, I have a team of engineers helping me.
In my opinion, this REP was an attempt to promote your company products using the names of NVIDIA and other hardware acceleration providers, the TSC members, and the ROS community. You tried to standardize your approach, stating the REP was informational, but talking about standardization elsewhere, including here and on the roboperf web page:
Standards
RobotPerf aligns to robotics standards so that you don’t spend time reinventing the wheel and re-develop what already works for most. Particularly benchmarks are conducted using the Robot Operating System 2 (ROS 2) as its common baseline. RobotPerf also aligns to standardization initiatives within the ROS ecosystem related to computing performance and benchmarking such as REP 2008 (ROS 2 Hardware Acceleration Architecture and Conventions) and the REP 2014 (Benchmarking performance in ROS 2).
It has been made clear that neither the TSC members, nor NVIDIA is backing up your standardization effort, and your reaction here is a strong ad hominem attack on all of us.
Also, here you are setting yourself as the defender/representative of the community (as if the companies you are criticizing here were not part of it). The community members will speak for themselves if they like your approach, simply by using Roboperf.
Victor, while I admire your drive for change and improvement, you may need to take a breath for a minute.
I quote the specific example of your own email below.
You claim you are giving a “summary of your proposals”, yet the summary you yourself give, is nothing more than either “criticism”, or “call to action”.
I dont see a single reference along the lines of:
“Here is problem X. And here is my specific, actionable proposal to FIX problem X”.
I have been working with Victor and I have been watching his efforts on this and other REPs for a while.
My take on this - the community and the TSC would definitely benefit from more transparency.
Which means 2 practical things:
publishing full video recording of the discussions of TSC meetings
making voting non-anonymous
From where I sit it would also be interesting to understand why Acceleration Robotics was not admitted to TSC a few months back. It does not look like it was for the lack of contributions…
And I disagree once again. I’ve presented various examples above, including the fact that the TSC has increasingly been making it very difficult. That it over-ruled the WGs and went beyond alignment. Into policy creation and into contradictions. TSC changed multiple times the policies to create a WG. TSC changed their mind multiple times about WGs categorization (who gets to be official, community, etc), which as of today, I still don’t understand why this is needed (as if TSC-created WGs are of more relevance). TSC members listed and unlisted things at their discretion at About Working Groups — ROS 2 Documentation: Rolling documentation without even pinging the corresponding WG leads. There’s not even the ROS 2 Hardware Acceleration Working Group listed in there today.
I applied early this year for the ROS 2 HAWG to become a TSC Working Group (and let’s highlight there’re existing TSC WGs led by non-TSC members official members, but observers). I did so, only to get pointed to the back-door and requested to “apply to the TSC as company for an official membership”. Of course this got rejected. I got %s, but no further input. Nothing.
So no, Ingo, TSC is not primarily alignment. Aligning would’ve been recongnizing the ROS 2 HAWG work and interest as well as its numbers and contributions. What happened is very different from alignment and that’s precisely why I started this rant. Because something’s very wrong with this TSC.
I understand that you may feel frustrated and believe that I am not directly acknowledging the suggestions to advance the matter technically. I stated my disagreement with the rejection and that such reponses/alternatives are not the right path as they’ll lead to benchmarking misunderstandings. It’s my right to do so, and I blame the current TSC. This is the forum to complain and discuss it.
I recognize that emotions can run high when we are passionate about a subject (and everyone knows that I am when it comes to ROS). It is everyone’s right to feel upset, but that’s not a reason to get personal as you did. Personal attacks hinder effective communication and prevent from collecting valuable feedback that could lead to a better outcome for everyone in this community. Additionally, while I have been accused of making ad hominem criticisms, my initial comments were carefully generalized. It was only after @gbiggs rundown of TSC members requesting further clarification that I particularized my comments providing some examples directed to their positions in the TSC (not that I dislike any of them or have a personal opinion. I just don’t care). So no, there is no “making myself out to be the victim here” Ingo, I’ve been taking responsibility for each and every one of my comments, responding with clarifications. Taking responsibility for my words is the opposite of playing victim.
And that is precisely the problem Jaime that I’ve remarked above. We need TSC members that are involved into the technology, to steer the technology in the right direction. You are not involved in the technology anymore, so ideally and in my opinion you should get someone involved in the technology from your side that can help steer the project. Otherwise, you could hire tomorrow another CEO and have her/him replace you in the TSC. After all, she’d be representing eProsima.
Wrong. Acceleration Robotics is a company producing semiconductor building blocks for robots using hardware acceleration. We make faster ROS components using hardware acceleration so of course it’s important for us to have an accepted consensus to measure performance. This helps us drive our product development and services. REP-2014 was an attempt to elevate the already existing consensus in the ROS community (which is using ros2_tracing) to a standard (yes, informational, but still standardized because it already is! The whole ROS stack is instrumented as such). Trying to reflect this (again, already existing consensus in the ROS 2 codebase) in a REP was the most logical step.
And just for clarity, again, REP-2014 and RobotPerf (an implementation of the REP) initiatives aimed to establish a common ground for robotics benchmarking, not to promote any specific company. They are built vendor-neutral (which is what’s scary for some companies) and that starts with ourselves getting out there, requesting feedback for multiple years (within the HAWG) and then writing some general consensus (REP-2014) and implementing it (RobotPerf). Btw, representatives of the TSC participated in some of these meetings. So you should be aware of this (but maybe choose to ignore it).
Also, note that similar to MLPerf, RobotPerf aims to become an industry standard by adoption. Placing ROS 2 at the center of this approach encourages roboticists to use ROS for performance evaluation, benefiting the entire robotics ecosystem. Attempting to leverage ROS’ successful community standadization efforts to generate an industry standard (RobotPerf) is nothing wrong or bad for ROS, quite the opposite. The informational and non-enforceable nature of REP-2014 was intended as a guideline, not a strict rule. Its rejection by the ROS 2 TSC is a missed opportunity to set common conventions that could have positively impacted the community.
Let me give you a direct example of the relevance of a “consensus in benchmarking”: in the last RobotPerf release we disclosed that we’re observing a very concerning behavior with Fast DDS when it comes to analyzing the latency of certain perception computational graphs:
This is not new and If you were more involved in the technology, you would probably be concerned by problems like this a bit more. Having REP-2014 out there would help us all align and agree on a general way to collect metrics, benchmark and compare.
Fair enough Phil, here’s a improved version of what I conveyed before, but now including actional proposals:
TSC Issue
Comment
Suggested action
Lack of Transparency
I criticized the lack of transparency in the TSC’s decision-making process, particularly in the evaluation of REP-2014, and other WG-related matters. I called for more openness and accountability.
All decisions by the TSC should be disclosed, including voters and votes
Non-Technical Rejection
I expressed frustration that the feedback for the rejection of REP-2014 was mostly non-technical, despite the proposal feedback being discussed, reviewed, re-iterated based on initial feedback, which we addressed in detail
TSC comments and feedback should remain technical. If they don’t have the expertise, they should either defer to the corresponding WGs or invite external independent experts (without conflicts of interest) that advise accordingly
Political Nature of TSC
I criticized the TSC for being more political and less technological, and questioned the value of the current TSC setup
TSC membership shouldn’t be voted by the TSC, it should be voted by the community as a whole (open poll to all members of the community), preferrably in an open/transparent manner (again to facilitate accountability). Also, TSC memberships should be renewed and voted periodically, e.g. every year. Politics and lobbying within the TSC should be rejected.
Lack of Strong Contributors
I criticized the TSC for missing strong contributors, key players, and individuals with a strong track record in the community.
TSC should be diverse in both gender and representation. Companies (Big companies, smaller and startups) as well as community (non-affiliated membership) representatives should get similar representation counts. The overall ROS community should have avenues/methods to raise concerns against TSC members, and methods to replace them if appropriate
Anonymous Lobbying
I expressed concern about anonymous lobbying (mostly corporates, and under confidentiality) and the confidential tone of ROS TSC discussions
Confidentiality should be an exception, and never used for TSC decisions
Need for Community-Centric Governance
I called for a change in the current leadership towards a more community-centric governance model
TSC community representatives should increase to match the count of company representatives and (regardless of their affiliation) should act as non-affiliated for what the TSC matters concern
Inconsistency in Standardization
I criticized the inconsistency in the TSC’s decision to reject standardizing a benchmarking approach in ROS 2, despite existing ROS codebase consensus, despite existing accepted implementations and despite the consensus reached among those involved in its development
Accept the community standards as community standards. Don’t reject what’s obvious (e.g. adoption of ros2_tracing) for tracing and metrics collection.
Lack of Accountability
I criticized the opacity and lack of accountability in the governance structure, and called for proper mechanisms to replace inactive or non-contributing representatives
All votes should be publicly disclosed, with the corresponding voters. Voters should be allowed to provide comments to clarify their position. All TSC members should be elected by the community, accountable for their acts and/or replaced given the right circumstances
Slowing Down of Innovation
I expressed concern that recent policies and events are slowing down the innovation process in the ROS ecosystem and discouraging many from continuing to contribute
Proactive contributions aligned with ROS interests should be encouraged. Not discouraged or stopped. TSC should carefully consider the technical value of each new initiative and its implications for the ROS ecosystem
So if you don’t code anymore, you are not involved in technology. Even if you manage a group of engineers, right? Fortunately, it is just your opinion.
Enclosing others hardware in a plastic box does not make you a semiconductor company. You need roboperf to become an standard to influence the market towards buying your boxes. It is obvious. And that is why you are pursuing this so badly.
I understand you are implying here NVIDIA. Do you really think a multibillion company can be scared because of this? Perhaps you should review your position in the market.
You did this already, and we didn’t answer, neither we are answering this time. It is your policy to try to market your products using other names. You have been doing this from long time ago with Micro-ROS, Vulcanexus, Safe DDS, Fast DDS… we decided long time ago not feeding you.
We answer to many performance “contests”, and we use several performance frameworks. We periodically run APEX.ai framework, the OSRF tests to get selected as the default DDS, our own framework (part of our CI system), nav2 performance (that’s a good real example), many customer scenarios in which performance is everything, github issues… But we will not consider your framework.
Bringing in this discussion the performance of our products (offtopic), my position within the company I represent, or my technical knowledge, it is a good example of trying to deviate the conversation by ad hominem attack.
So, I reiterate: You tried to standardize your approach, subverting the process by stating in the REP was informational but talking about standardization elsewhere and using other’s names to imply consensus (NVIDIA) without their consent.
Now, as it was rejected, you attack the TSC members and its governance. I think at this point is clear to anybody why you got your REP rejected.
Jaime, while I do not wish to engage in public fights with you or a discussion about Acceleration Robotics, I feel compelled to address the attacks and misunderstandings in your response. It seems there is a fundamental misunderstanding on your part about my role and the work we do at Acceleration Robotics.
Your characterization of our work as merely “enclosing others hardware in a plastic box” (referring to ROBOTCORE RPU, an open community-oriented design we disclosed as part of our work in the HAWG[1]) is not only incorrect but also demonstrates a profound lack of understanding of the technology and the broader robotics ecosystem. Semiconductor building blocks refer to Intellectual Property (IP), or hardware designs that then turn into hardware (and or silicon, depending on whether they become soft-cores or hard-cores). Your own products run on what was once a soft-core, only to become a hard-core. The whole robotics ecosystem (and hopefully spearheaded by ROS) is going to benefit significantly from domain-specific compute architectures. That’s what we’ve been doing for a few years now. That’s what many are doing now.
Examples of some of our released products include ROBOTCORE Transform or ROBOTCORE Perception, which are ROS 2 API-compatible and give you a computing performance increase. You’re welcome to check those (or have your team look at them) and send feedback if you wish.
You’re once again misunderstanding (purposely?). I will repeat it: because “you are not involved in the technology anymore, you should get someone involved in the technology from your side that can help steer the project (participate in the ROS 2 TSC)”. I’m not making a statement about you personally, or about technology in general. I’m questioning your capabilities as a ROS 2 TSC representative and how much that brings to the community.
Your decision to dismiss our framework outright without consideration is disappointing, but not surprising given your apparent lack of understanding of our intentions and the work we do. Accusing me over an over of ad hominem attacks and attempting to deviate the conversation is a diversionary tactic that does not address the substantive issues at hand.
Your attack on my position, and our products is very unprofessional. As a TSC representative, it is your responsibility to represent the interests of the ROS community, not just your own or your person or your company’s. Your response, however, seems more focused on attacking me and my company rather than addressing the substantive issues at hand. Again, asking for more accountability from TSC members.
Your comments about NVIDIA and your accusation of us using other names to market our products are ridiculous, baseless and unfounded. We have always been transparent about our collaborations and references to other projects and companies are made in the appropriate context.
This (discussion) is about the ROS community, not the overall market. I’ve been extremely critical myself towards the attitude of companies like NVIDIA over multiple threads here in Discourse, calling for technology alignment (as opposed of forks and re-marketing), calling against their lock-in approach by forking and changing APIs, calling for community-aligned benchmarking (using what’s already a standard, but not qualified to become a REP, that is ros2_tracing) and also complaining about those times wherein NVIDIA forked packages and changed their license to make it less commercially friendly (packages many of us here have contributed for a decade).
This is one of those things TSC members like you should be looking at (and btw, as acknowledged above, we got positive “reactions” out of these past discussions).
And again, I reject your view. It’s not my approach, it’s the community approach. Not because I say so, because it’s been embraced for years now. The whole ROS 2 stack is instrumented with ros2_tracing which is the basis of REP-2014 (again, something you’d know if you were closer to technology), which makes me confident that it should get standardized, and which triggered this whole rant. There’re even DDS implementations that have been instrumented with ros2_tracing and/or LTTng (e.g. CycloneDDS) giving robotic engineers the capability to better understand their dataflows. It’s important for you to understand that this is not me being right, it’s what’s out there, what’s adopted. When/if your group @Jaime_Martin_Losa decides to align to the trend (and instrument your DDS solutions to align with LTTng/ros2_tracing), you’ll be aligning with the community, not “feeding me”.
By the way Jaime, you’ve repeatedly pointed out that there’s no consensus. But there is, even with NVIDIA. As I explained above, the rejected the use of ros2_tracing (grey-box performance benchmarking) and instead decided to re-implemented ros2_benchmark (black-box performance benchmarking). We studied their approach, forked it to improve it and integrated it in both REP-2014 and RobotPerf (empowering you to do both). So yes Jaime, we’ve worked towards consensus from a technology point of view. And I think we have it.
From your words, It is clear that the rejection of the REP has more to do with politics and personal biases than the merits of the proposal. Your responses @Jaime_Martin_Losa only reinforces this perception. It is disappointing to see this level of unprofessionalism from a TSC representative. Instead of attacking those who are trying to contribute to the community, perhaps it would be more productive to focus on constructive feedback and working towards solutions that benefit the entire community.
Repeating myself, I recognize that emotions can run high when we are passionate about a subject. I’ll try to be more constructive re-iterating those proposals I believe are necessary for situations like this to be avoided in the future (small edits included on each proposal stated before, hopefully improving them):
ROS 2 TSC Issue
Comment
Suggested action
Lack of Transparency
I criticized the lack of transparency in the TSC’s decision-making process, particularly in the evaluation of REP-2014, and other WG-related matters. I called for more openness and accountability.
All decisions by the TSC should be disclosed, including voters and votes
Non-Technical Rejection
I expressed frustration that the feedback for the rejection of REP-2014 was mostly non-technical, despite the proposal feedback being discussed, reviewed, re-iterated based on initial feedback, which we addressed in detail
TSC comments and feedback should remain technical. If they don’t have the expertise, they should either defer to the corresponding WGs or invite external independent experts (without conflicts of interest) that advise accordingly
Political Nature of TSC
I criticized the TSC for being more political and less technological, and questioned the value of the current TSC setup
TSC membership shouldn’t be voted by the TSC, it should be voted by the community as a whole (open poll to all members of the community), preferrably in an open/transparent manner (again to facilitate accountability). Also, TSC memberships should be renewed and voted periodically, e.g. every year. Politics and lobbying within the TSC should be rejected.
Lack of Strong Contributors
I criticized the TSC for missing strong contributors, key players, and individuals with a strong track record in the community.
TSC should be diverse in both gender and representation. Companies (Big companies, smaller and startups) as well as community (non-affiliated membership) representatives should get similar representation counts. The overall ROS community should have avenues/methods to raise concerns against TSC members, and methods to replace them if appropriate
Anonymous Lobbying
I expressed concern about anonymous lobbying (mostly corporates, and under confidentiality) and the confidential tone of ROS TSC discussions
Confidentiality should not be part of the TSC on a general basis and never used for TSC decisions, or to justify the TSC. Members can establish confidentiality outside of the TSC if needed. Nobody should be put in the position of “hey, resign because you’re breaking confidentiality”. Transparency should be an inherent part of what the TSC does.
Need for Community-Centric Governance
I called for a change in the current leadership towards a more community-centric governance model
TSC community representatives should increase to match the count of company representatives and (regardless of their affiliation) should act as non-affiliated for what the TSC matters concern
Inconsistency in Standardization
I criticized the inconsistency in the TSC’s decision to reject standardizing a benchmarking approach in ROS 2, despite existing ROS codebase consensus, despite existing accepted implementations and despite the consensus reached among those involved in its development
Accept the community standards as community standards. Don’t reject what’s obvious (e.g. adoption of ros2_tracing for tracing and metrics collection, when the whole ROS 2 stack is already instrumented accordingly).
Lack of Accountability
I criticized the opacity and lack of accountability in the governance structure, and called for proper mechanisms to replace inactive or non-contributing representatives
Better accountability by recording meetings. Also, all votes should be publicly disclosed, with the corresponding voters. Voters should be allowed to provide comments to clarify their position. All TSC members should be elected by the community, accountable for their acts and/or replaced given the right circumstances
Slowing Down of Innovation
I expressed concern that recent policies and events are slowing down the innovation process in the ROS ecosystem and discouraging many from continuing to contribute
Proactive contributions aligned with ROS interests should be encouraged. Not discouraged or stopped due to lobbying and market interests. TSC should carefully consider the technical value of each new initiative and its implications for the ROS ecosystem. If in conflict, other companies’ vision about such new contributions should be discounted
Inequitable Working Group Establishment
I criticized the current exclusion and classification among Working Groups (WGs), as we are all part of the same community and represent the same interests, whether on the TSC or not.
Establish more homogeneous WGs that do not exclude or classify members based on their affiliation with the TSC or any other organization. All members of the ROS community should have equal opportunities to participate and contribute to the WGs.
Again, you’d know this if you were technically competent and or would’ve been tuned to the Hardware Acceleration working group activities. ↩︎
After discussing this within the moderator team, we’ve decided to close this thread.
We think it is important to have open and honest discussions about how the ROS project is governed, and be responsive to the wishes of the entire community. However, this particular discussion has degraded into ad hominem attacks, hear-say, and last-wordism that is anathema to a productive conversation. Litigating past TSC discussions isn’t going to get us moving in a positive direction. Positive discussions propose better approaches on a go-forward basis. Moreover, we’re not going to retroactively change any past TSC decisions by fiat in a Discourse thread. Finally, and most importantly, discussions about changes to project governance need to happen in a neutral and dispassionate manner; the quieter members of the community are not going to weigh in during a shouting match.
From where I am sitting, chairing the TSC, I am always happy to clarify the discussions that have occurred, and I would also ask that there be an assumption of good faith with respect to TSC operations. Finally, if this is the type of work that interests you, TSC Community Representative elections are also going to kick off the week after next, and everyone is welcome to apply.
For the time being I am going to ask everyone to take a break and find more constructive means of addressing project governance.