ROS 2 TSC Meeting Minutes 2023-08-17

ROS 2 TSC Contribution Report 2023-08-17.pdf (127.6 KB)

ROS 2 TSC Meeting Minutes 2023-08-17

  • Attendees

    • TSC Members
      • eProsima / Jaime Martin Losa
      • Community / Francisco Martin Rico
      • OSRF / Geoff Biggs
      • ROBOTIS / Will Son
      • Sony / Tomoya Fujita
      • Picknik / Henning Kayser
      • Apex.AI / Lyle Johnson
      • Canonical / Jeremie Deray & Mirko Ferrati
      • ROS Industrial / Matt Robinson
      • Devcom / Jerry Towler
      • iRobot / Alberto Soragna
      • Toyota Research Institute / Ian McMahon
      • Foxglove / Adrian Macneil
      • Intrinsic.ai / Katherine Scott
      • Amazon / Aaron Blasdel & Matt Hansen
  • TSC Observers

    • Bence Magyar
    • Steve Macenski
  • Absent (Please move to TSC Members Line):

    • Intel / Harold Lang
    • Microsoft / Lou Amadio
    • Intrinsic.ai / Chris Lalancette
    • Samsung / ???
    • Wind River / Andri Kholodnyi
    • Zettascale / Angelo Corsaro
    • Community / Patrick Musau
  • Excused

    • Bosch / Christian Henkel
    • Community / Oliver Michel
  • Preliminaries

    • ROS Answers officially in maintenance mode.
    • ROSCon talks go out today – please help spread the word!
    • Gazebo Harmonic Tutorial Party September 1st, 2023
    • Who is planning to attend IROS?
      • Looking for a third sponsor for a ROS meetup.
      • TODO: follow up with Foxglove
    • [Jeremie Deray| 5min] Change of Rep at Canonical
      • New rep Mirko Ferrati
      • TODO: Update ROS Documentation
  • Old business

    • [Kat|5min] REP-2014 Results
      • REP-2014 Not Accepted.
        • Feedback will go after meeting
      • 15 votes, 3 for 12 against
      • TODO: Kat follow up with REP Authors

  • [Kat|10min] TSC Meeting @ ROSCon 2023

    • 13 Responses: 12 will be at ROSCon
    • 83% want Wednesday, October 18th, 2023 (workshop night)
    • May not be a working meeting
    • TODO: Kat will send out invite for event / survey for event.
    • TODO: Kat to move October meeting to October 12th
    • Proposal
      • Schedule dinner meeting for October 18th at 7pm
      • Will not be a working meeting
      • Will need a final headcount.
      • Move official TSC meeting to October 12th, 2023
  • New business

  • Recurring business

    • Next ROS 2 distro release
    • Working groups [<=5 mins each]
      • [Bence] Control

        • Robotics SE: Anyone has enough reputation to create ros2_control tag?
        • Fixed issues regarding controller update rate
        • Controllers are now sorted internally according to inter-dependencies (plus reported accurately in CLI)
        • Hardware components
          • Initial states can now be set
          • Additional CM checks for HW initialization
        • Python? Python! Bindings may happen…
        • New range sensor broadcaster
        • diff_drive_controller new tf namespace parameter
        • joint_trajectory_controller:
          • Reject effort goals (not supported)
          • Reject nonzero terminal velocity
          • Explicitly set hold position (feature flag)
          • Hold on time violation
          • 200% tests
      • [Martin Losa] Embedded

        • No update this month due summer vacations.
      • [Kayser] Manipulation

        • Open-Sourced new Kinova driver - ros2_kortex
          • Several MoveIt tutorials now use Kinova hardware
        • GSoC: large Servo refactoring merged - consolidating twist command interface, cleaning up a lot of internals
      • [Woodall] Middleware

      • [Macenski] Navigation

        • Fix velocity smoother deadband not being respected after another recent update
        • Allow the Nav2 parameter file to include parameter file substitutions within the yaml file
        • Include error checking / handling on invalid inflation layer parameters for the optimized collision checking in MPPI or Smac
        • Added additional debugging visualizations to Smac Planner to help with footprint swept volume analysis
        • Release an Iron, Humble
        • Added new collision checker node within the collision monitor package for notifying of zone breaches rather than directly taking control over the robot
      • [Andrei] Real-Time

      • [Deray] Security

        • Meeting canceled due to summer break
      • [Orlov] Tooling

2 Likes

Rep-2014 link goes 404. Is that meant to be [REP-2014] RFC - Benchmarking performance in ROS 2?

No, that link actually goes to the formal vote in the private TSC Discussion channel. That is a helpful reference link though,

1 Like

Thanks @Katherine_Scott for leading this all the way into its resolution, from that standpoint, I appreciate it and thank you and @fmrico for bringing this into the TSC discussions.

For transparency, here’s the feedback provided by the TSC concerning REP-2014 and that led to to its rejection:

Feedback provided by the TSC concerning REP-2014
  • It does not include other alternatives.
  • The proposal includes recommendations that are valid, but REPs are for things that ROS does or will do holistically. That is not feasible and so I will vote against accepting the REP proposal. In addition, I don’t see why it needs to be a REP. This info could just as well be part of the documentation and doing so does not detract anything from the messaging. If anything it gives more flexibility and chance to evolve.
  • The proposed REP is composed primarily of informative information such as what would be found in a tutorial rather than a common specification. The authors should propose adding a benchmarking tutorial to the ROS 2 documentation instead.
  • There is definitely a lot of value in writing down some procedures and best practices for running benchmarks with ROS 2. However, the goals and tools used for benchmarking can vary wildly based on the exact thing being measured. And these tools will evolve over time (which is acknowledged in this REP in the sentence “The guidelines in here are intended to be a living document that can be updated as new information becomes available.”). Because of these factors, I believe that this document doesn’t really fit into the REP process. Rather, I think it would be better as a document that lives on https://docs.ros.org
  • I’m unclear on the benefit of this being a REP rather than a blog post. Besides, concerns were raised about the level of community agreement around this.

I’m obviously not happy about the rejection. Most aspects in the feedback were discussed and addressed (see https://github.com/ros-infrastructure/rep/pull/364) and despite skeptics, there’s a general acknowledgement of its value. An open discussion would’ve been more constructive. What I’m most frustrated about is the fact that feedback is mostly non-technical, but comes as a rejection from the ROS 2 TSC (which stands for ROS 2 Technical Steering Committee :wink: ).

This unfortunately confirms once again that in my view:

  1. working with this TSC has been “complicated” for a while now. There’s increasingly less and less technology, and more politics.
  2. that the current setup of TSC “thought-leaders” (most of which have never contributed to a REP, or ROS with relevance) adds very little value, and
  3. raises again various questions about the value of the current TSC setup, missing strong contributors, key players and individuals, with a strong track of record in this community (but of course, not backed by X company, and its aligned interests).

I can’t help but continue being surprised everytime someone totally unknown (and without relevant ROS contributions) is appointed by company Y as new TSC representative. This is a reflection of the status of affairs. Seeing that distorted now increasingly, conveys a very strong message against contributing (and we’re all seing the huge slowdown in Discourse engagement/involvement).

I’ve heard a few times already promises concerning “relevant changes” in the ROS community governance model. I’d like to see that happening so that we stop seeing “anonymous lobbying” for once. Also, please, let’s stop the “confidential” tone being set around the ROS TSC discussions for a while now. Transparency is one aspect that inspired many of us, and one of the key values of how this community (and technology) was built by many of us.

For what’s worth, It might be a good idea to remind TSC members of the differences between Informational REPs and other types as described in REP-0001:

An Informational REP describes a ROS design issue, or provides general guidelines or information to the ROS community, but does not propose a new feature. Informational REPs do not necessarily represent a ROS community consensus or recommendation, so users and implementors are free to ignore Informational REPs or follow their advice.

As with various other topics within the feedback, this was already clarified in the public discussion.
Let’s be more public and transparent for what concerns forward actions from the TSC please.

5 Likes

Victor, I get your frustration, and I see that you’ve attempted to address many of the concerns. My reading of the discussion, is a bit different than yours, though. Firstly, it has occurred in the context of the PR and was thus very transparent. I don’t know what you think goes on in the TSC, but its really mostly alignment – seeing whether all the participants agree or whether more discussion is needed, and if substantial discussion is considered necessary, it happens outside of the TSC in the working groups and in PRs. Last, but not least, I think enough of the TSC participants have a strong background in the community, and the others provide important external views while coming from companies who have a strong ROS involvement. Therefore, I reject your characterization of the TSC. I think the governance model is working well.

Regarding how the discussion went: Various people have brought up the same points over and over, and yes, you have replied to them, but I didn’t get the impression that people were really convinced. And that was really a common thread, and it wasn’t just one group of people. And I don’t mean to put this on you, I certainly thought some of your points were very valid, and others were certainly an opinion with a foundation, even when it wasn’t uncontroversial. However, at the end of the day, I don’t think we can say that there is a consensus, yet.

And that what it really boils down to: A REP has a certain meaning – people see them as consensus and accepted guidelines, regardless of whether they are just “Informational” or not. Things have moved on since REP-00001 and when all the other REPs are a consensus (or at least no contested), expectations change.

Therefore, I urge you to not over-interpret this decision and the proposed alternatives are good ones, IMO. They will achieve almost all of the same goals that a REP would have achieved, maybe even better. Sometimes, you just have to agree to disagree and move forward in a way that is acceptable to all. I think the TSC decisions enables us to do that.

6 Likes

No Ingo, this is incorrect and that’s one relevant point of my criticism.

The discussion about the evaluation of this topic from the TSC started a few months ago and wasn’t public. I presented this in an unrecorded, private and direct manner to the TSC as per their request and there were no comments nor questions at the time, but many just afterwards, the moment I left the room. It was only due to some frustrated TSC members, who disagree with the current community-unfair approach, that I got to know about some of these comments.

Some of these comments were really unacceptable. This confidentiality is really firing back on the community for relevant decisions. This is an open community and TSC members should stand behind their words and organizations with transparency. Note I’m not against confidential information being shared, but if of relevance for a decision should be made public. This is the least an open community should demand in my view.

For example, in this particular case, in these private (“confidential”) conversations while evaluating REP-2014, the “authors of the REP were criticized” as not representative enough. Specially it was pointed out that other than myself, including your yourself Ingo (@Ingo_Lutkebohle) were “added to sell this better”. But both you and I @Ingo_Lutkebohle know that you yourself asked me to be listed as an author, and I accepted because besides the feedback you exchanged for REP-2014 privately, we’re all very well aware of your support and contributions to the tracing and benchmarking fields. Note this situation would’ve been very easy to address if made transparently, but not.
Another example issue in these confidential discussions was the criticism that “It does not include other alternatives”. It was requested months ago for REP-2014 to be more inclusive, so we went ahead modifying the language and incorporated ros2_benchmark (originally contributed by NVIDIA) into REP-2014. Of course the TSC member who made again this remark is unformed and would’ve benefited from a public discussion.

The fact that there’s no historic transparency (with the community) about the TSC in-take and application process, or that some companies enter through the back-door while others have to wait for months for a private resolution are yet again reasons why this confidentiality is against the interest of the ROS community. Sure we see someone important in the minutes from time to time, but what about the 10+ startups that keep applying and are neglected stealthily despite their contributions? And please don’t bring up community representatives here, TSC rejected changing them when pointed out their lack of participation. There’s even one of them who is not showing up. The last batch of applicants were of real quality and I would’ve taken them all to balance the corporates in the TSC today. We are past the early days of industry engagement. We need a community centric governance.

I’m worried about the current leadership, I don’t believe it works, and I believe it needs to change towards a community-centric establishment wherein topics like the documentation, the ROS fragmentation and/or crucial aspects of the technology are treated more, and politics left aside for the market to decide.

And again, an Informational REP is meant to lead the way, because consensus takes time, and iterations. That’s why this path was selected. REPs can always be updated or discarded afterwards.
This decision (a rejection) is a mistake that we all will suffer. TSC voted 3/12 against standardizing a benchmarking approach in ROS 2. They did so despite 10+ implementations (some listed here) of such a thing existing. And more to come!

Everyone should note how confusing it is that not even the TSC wants clarity in how measurements shall be performed and compared. This situation presented a big issue in other community (for example the Machine Learning community). If each one of us comes up with its own way of measuring things in ROS 2 (as we know happened in the so called “DDS wars” ) it’ll be very hard to accept anyone’s claims of performance.

2 Likes

Victor, firstly what you described about the TSC interaction does not really refute what I said about it being mostly alignment or the fundamental discussion happening on the PR.

Secondly, I think you’re blowing this completely out of proportion. I’m sorry to say this, but it’s really no wonder some people have become wary about working with you, if you react to being rejected by spouting unsupported allegations against the entire governance structure.

As I said before, we have many ways of improving measurement and benchmarking in the ROS community without going the REP route. The TSC has suggested as much in their detailed response. Lets focus on that and not waste any more time tilting at windmills.

Ingo, your attempt to dismiss my concerns as overblown and accuse me of spouting unsupported allegations is both disingenuous and hypocritical. The opacity and lack of accountability in the governance structure are glaringly obvious, and your dismissal of these concerns, given your position and relationship with Bosch, smacks of self-interest. It’s clear that there is a vested interest in maintaining the status quo, even if it is at the expense of the broader ROS community.

Your decision to deflect by questioning my reaction and making it personal, rather than addressing the legitimate concerns I raised, is a textbook example of bad faith argumentation. It’s precisely this kind of behavior that discourages open and constructive dialogue and makes people, including myself, wary of the current leadership and governance structure.

While I agree that the REP route may not be the only avenue forward, it’s a missed opportunity. The TSC’s decision to reject standardizing a benchmarking approach in ROS 2, despite existing implementations, is a short-term view that will have long-term consequences. It’s akin to not standardizing railway track sizes because there are already many in use - logical in the short term, but haunting in the long run.

And on a lighter note, it seems that acting like a #bosch really got into someone’s head!

@vmayoral

thank you for sharing the opinion.

personally i believe that it is really brave to speak this out. (i really mean it.)

all these opinion and feedback should be taken into consideration.

i have a question though,

various questions about the value of the current TSC setup, missing strong contributors, key players and individuals, with a strong track of record in this community

I can’t help but continue being surprised everytime someone totally unknown (and without relevant ROS contributions) is appointed by company Y as new TSC representative.

based on what data or information are you saying this? like just in general or statistics?
i think there are many types of actions to be judged as contribution, some can get attention, but some cannot…

if necessary, you can take me (TSC, Representative of Sony Corp) as an example. i am happy to stand up on this with you and for ROS community.
if you are uncomfortable with this question, i also understand that. please ignore this, and you already know that is NOT my intention.

i am trying to be open and neutral, i do want the ROS Community to be comfortable for everyone. (including me :smile: )

thanks,
Tomoya

@vmayoral I think you realize that it is inconsistent to position your proposal as being “just an informational rep” and at the same time expecting it to “standardize the benchmarking approach”.

To me it is rather clear that for a standard to be accepted, one needs to reach consensus first. Trying to push a standard without consensus, by writing a document not at all in an informational tone and just specifying informational “clearly” at the top, is bound to have it rejected.

2 Likes

@JRTG Thank you for the feedback. If it was so obvious that it was bound to be rejected due to issues you mentioned, shouldn’t these have been brought in the public discussion. We have always been open to criticism and feedback and thus the rep has gone through multiple iterations since first draft is September 2022. There have been various contributors/commentors on the REP and they have been respected with their feedback and changes made. The community thus had a consesus with the REP and its value. Very few TSC members were actually gave their feedback on the REP(which again was taken into account) but most did not. The rejection reasoning is not clear enough and an open discussion with the TSC to account for their feedback and transparency is what we are asking for!

1 Like

Adding on to @PratNag’s point, there’s been a pretty strong consensus in the Hardware Acceleration Working Group about this with participation even from big silicon vendors. Will request and highly encourage to participate in the HAWG meetings to see the same!

1 Like

@tomoyafujita, thank you for your question and for being open and neutral. To answer your question, my reasoning is mostly based on 1) past experiences with/in the TSC (you may remember I was one of the early members of the TSC), 2) plenty of discusions with companies operating in the ROS space whose contributions didn’t move forward or got rejected while trying to participate in the TSC more actively (and with contributions in mind) and 3) digesting the information publicly available from the TSC Meeting Minutes for the last few years (I don’t claim to have read everything, but a pretty decent amount to generalize). This information (3)) lacks details and transparency for the community (which a few get to represent and discuss monthly in the TSC, sure, but still, the TSC is the ROS project/community’s TSC and each one of these representatives should act responsibly to it). Again, there should be accountability. I think demanding full transparency with regard relevant votes/decisions is very fair.

As I was hinting above, many current TSC members have not authored a REP or been exposed to or ruled by the policies they are setting, which have changed rapidly in recent years. Due to this policies, despite ROS being an open project, it has become increasingly difficult to contribute, especially with new ideas, unless you are in the TSC or “own” the piece you aim to contribute. We need a more diverse TSC committee that encourages and includes more community representatives, at least as many as corporate ones, and who actively engage with the ROS community’s issues and needs. If representatives do not pursue the community’s interests with evidence or are inactive, there should be proper mechanisms to replace them.

Let me give you a practical example: REP 2007 started sponsored by a TSC member (great contribution, by the way) on Jan 28, 2021 (PR) and it took 4 months to merge it. REP-2008 started on August 25, 2021. After much (much!) insisting, various TSC presentations, TSC discussions and TSC reviews, it was pushed back to the ROS 2 Hardware Acceleration Working Group (who was the original proponent) and approved in 16 months (as a “Draft”). I’m happy this went through but it was a terribly painful process, which involved explaining things over and over to TSC members who were not technically qualified on the topic. Before REP-2008, no REP proposal went through such scrutiny. I have more examples if needed.

In recent years, the TSC has overruled, especially with new policies that were not publicly discussed or accepted but introduced on-the-go. These policies and recent events are slowing down the innovation process in the ROS ecosystem and discouraging many from continuing to contribute.

@JRTG as some have pointed out above, you’re missing information. I encourage you to review the ROS 2 Hardware Acceleration Working Group (HAWG) resources and particularly, the recordings.

Consensus was reached among those that decided to get involved in this work. This REP has been discussed throughout those meetings for a very long time (since Sept 29, 2022). Many of these meetings were very well attended 80+ participants and included dozens of participants from the ROS community, including experienced deverlopers and newcomers. This REP also encouraged repeatedly the participation of silicon vendors (who are generally very interested in benchmarking) and received reviews from various of them. We particularly insisted on NVIDIA to join/support the effort, but instead, they decided to go independently and disclosed ros2_benchmark instead of aligning and joining efforts.

Despite NVIDIA not collaborating, in an attempt to be more inclusive (and following from prior TSC input), we went ahead and within RobotPerf (which is a reference implementation of REP-2014) included ros2_benchmark as one of the options, supporting NVIDIA’s take. We’ll be publishing an article about this shortly.

Accordingly, I don’t accept the inconsistency you remark. We’ve tried really hard to reach consensus, even when ignored.

2 Likes

Open to criticism and feedback != alignment and consensus.

This has not been our experience. Much of our feedback has been attacked or ignored.

Our monitoring of HAWG has been used to imply, as it is in this thread collaboration and agreement. This misrepresents a lack of alignment and consensus hence we stopped attending HAWG.

Our work on bringing accelerated computing to ROS 2 requires we regularly measure performance. We could not do that with the requirements as stated in the rejected REP. We released tooling for ROS 2 benchmarking in open source and ROSBag datasets in Creative Commons v4 Commercial to provide benefits of the tooling and choice to the community.

Perhaps you are overlooking your victory :medal_military:. Driven by your initiative, a commercial quality tooling for benchmarking accelerated computing in ROS 2 is available in open source.

2 Likes

@vmayoral First, to be clear: I have no personal interest in benchmarking and I do not have an opinion whatsoever about which approach to ROS 2 benchmarking is better or preferrable or whatever.

That being said:

You claim e.g. “consensus is reached” and “Most aspects in the feedback were discussed and addressed”.

I have read the REP proposal, the PR discussion and the Discourse thread.

My conclusions are:

  • Either the REP is intended to standardize the benchmarking approach for ROS 2, in which case:

    • It should not be an informational REP, and
    • At the very least it should address these and these comments, which it does not.
  • Or, the REP is intended as an informational REP, but

    • It is not at all written in an informational tone,
    • Not even this small request was applied to clarify the informational status.

I take note of your ongoing quarrel with Nvidia, and I take note of your complaints about the TSC procedures. Maybe you have a point, maybe you don’t. I am not voicing an opinion on that, I am voicing an opinion on this REP proposal. This proposal:

  • Was not intended to seek a common ground,
  • Does not comply to a Standards REP,
  • Does not comply either to an Informational REP.

So is it is only logical that it was rejected.

Alignment and consensus with whom exactly? That is where our question lies. You raised your voice and were not happy that some of your asks were not incorporated and thus it is understandable for you to not be happy with REP. But the REP is being voted by TSC most of whom have not been a part of the process. Of Course all community REPs should not be accepted and there should be a due process for quality control, but the current procedure to get a REP passed is just to make sure that all TSC members are appeased which for a community driven ecosystem does not make sense.

I have huge respect for most of the TSC members and organisations and there needs to be industry involvement in this process but shouldn’t the community get an equal say as well? Apart from REP 2014 which we initiated and got through after a lot of difficulties, none of the REPs accepted to my knowledge have been community driven. We are not even given a chance to put forward and have an open discussion with TSC as we are not a part of it, thankfully @fmrico has been supportive and brought forward our REP to the TSC.

We would have been more open to address the suggestions if there were more active contributions rather than just comments. Lack of resources on our end makes it difficult to do incorporate all requests.
The comments that you mentioned if you continue reading you can see Steve replying back accepting the position of the REP as informational.

This was immediately replied to here mentioning that it is very clearly mentioned at the start itself that this is an informational REP. See REP 0101 for example and other informational reps. We are just following the standard ways on how REPs are written. None of these REPs were scrutized or asked to make it make it explicitly clear to everyone over and over again by mentionining in 10 different more places that this is an Informational REP.

You are still bound to have your opinion to reject it and I respect that but downplaying that we have not accounted to proposed changes whatsoever and us being critical of TSC and it’s processes is what I don’t agree with.

Anyways I don’t think we should put more of our mutual energy over this. I am sure things will change and something better comes out of this in the future! In the end we all want the ROS Community to succeed!

On the Specific REP

I definitely acknowledge the value of this work, full stop. However I’ve expressed in the PR reviews I’m not sure an REP is the right place for this work in no uncertain terms. Honestly I think an academic paper with a more user-friendly blog post describing this method / process as a reference is the right solution. An REP could then follow after gaining sufficient traction and refinement over years until its reached effective consensus.

While the informational REP does meet the intent of this work (e.g. provide information) that doesn’t mean that that information doesn’t need consensus across a broad spectrum of the community or that because it is submitted as an informational REP that that is really the right type of REP for the class of work described. I think it needs to better fit in either camp:

  • Detail to be totally prescriptive as would be required of a standard covering all major needs (and be proposed as a standard)
  • Consolidation of opinions and general consensus across the more broad community than a single corner for informational purposes around a software package so it can omit the prescriptive details – over years of usage an refinement. If its not the “defacto standard” or “there is no genuine alternative”, its not there yet.

To a degree, the continual pushing is what makes it somewhat a problem and creates the tension. You also have to convince folks of your ideas, not just address the specific technical concerns that are articulated as action items. Then once running out of action items, demanding a vote to ratify.

Baking time is not uncommon in these kinds of documents and delays on the order of year(s) should be expected to give enough time for consensus to be reached / fully appreciated. I have REPs open for > 1 year and I’m pretty certain they’ll get there eventually, but baking time is part of the process. Every ~6 months when I kick the tire, I can feel the gears moving my direction and new thoughts brought up which make it better. I think this is an important point to make. The gears of standardization turn slowly and its hard to speed up “this method is the best method” until you have years of use under the community’s belt.

Just because there isn’t consensus today, doesn’t mean that there never will be.

I think this is perhaps where we lose each other. That’s certainly a concentration of important folks, but not the exclusive set of them.

On the TSC

Sitting in those meetings, I don’t think that’s really true. I think there’s less and less of everything as we wait for whatever the new system that is being put together by OSRF to be put together :slight_smile:

I can hardly look at the list of TSC members on the attendees list above and note someone that’s not contributed to ROS with relevance.

I reject in entirety your characterization of the TSC operations. I won’t say the TSC doesn’t have other issues, but politics and confidential lobbying isn’t one of them (currently). And certainly no one that I’m aware of is politicking or lobbying about REPs now, in the past, nor expected in the future. It won’t surprise anyone that is a more hot topic when default DDS vendors are being discussed, but little else. We’re not an evil cabal in smoky back rooms confidentially gate keeping the ROS community. Frankly most meetings little else happens more than getting a run-down on everything that’s going on and @Katherine_Scott giving a dirty look to all the companies not filling in their contribution reports.

That’s absolutely ridiculous. If that were true, I’d be a TSC member and not an observer there to present the Navigation Working Group summaries. There are a handful more comments like this that are outlandish but I don’t think merit further asks for clarification, because I’m sure there is none to give rooted in reality.

This seems to me just a general airing of perceived grievances not substantively backed by the reality of things. If you have specific and productive policy guidelines that you’d like to propose, I think there’s a body of folks that are open to hearing about it. But if you just want to complain and attack, that’s not going to change anything (I know, I love complaining :slight_smile: ).

3 Likes

This is going to be a long post, because Victor has made a number of points that I want to address in turn for the benefit of the whole community.

On the feedback from the TSC

TSC members who felt that their feedback was relevant to improving the proposed REP document itself provided commentary on the PR. Other TSC members may have felt that their informed opinions were less useful as feedback and more useful for allowing all TSC members to make a fully-informed vote. These opinions don’t necessarily need to be conveyed to the authors of a proposed REP nor made public. TSC members are not required to comment on or contribute to a proposed REP in order to take part in the review process.

Furthermore, the feedback given by the TSC as part of the voting process is an ad-hoc process change made by @Katherine_Scott, the TSC chair. This change was made because it is important to have more transparency on why TSC members voted the way they did, allowing REP authors to potentially address those concerns and resubmit the proposed REP.

The feedback provided after the vote by the TSC is in agreement that the TSC would prefer the content you are proposing to be in the documentation rather than as a REP. This is probably in part based on the claim in your proposed REP that, although you state that the proposed REP is informational, the “guidelines in here are intended to be a living document that can be updated as new information becomes available.” REPs are explicitly not meant to be living documents. They are set in stone once they are accepted and published, and changing a REP means making a new REP and deprecating the old one. This is necessarily a long process. On the other hand, updating a documentation page is an easy and common process involving a simple PR.

On informational REPs

Although the original REP describing the processes around the REP (written in 2010, since when a considerable amount has changed in our community) does include the informational REP concept, this is not a requirement for proposed informational REPs to be accepted, particularly if there may be a better way to present the information.

Moreover, informational REPs are still meant to contain information that is broadly agreed upon. Reading over the PR discussion and relevant threads on Discourse does not convince me that there is a broad agreement, let alone a consensus. Perhaps the TSC members felt the same.

I also note that you have mixed up “informative” and “standardised” in multiple places in your posts. You state that the REP is intended to be informative, then make statements such as “TSC voted 3/12 against standardizing a benchmarking approach in ROS 2.” The REP cannot be both informative and set a standard at the same time. If your goal is truly to get the information out there so as to inform users of ROS one way to do benchmarking in order to encourage more of it to be performed, then a page in the documentation would be sufficient. An informative REP shouldn’t hold significantly more value for you, especially given its lower visibility.

On the REP process

I agree that the REP process needs an overhaul. Fortunately the OSRF is working on a new REP process as part of its year-or-more-long transition away from the old BDFL project governance model that the OSRC represented. We will call for community input into this at the appropriate time.

However, we can’t revise the process and apply it retroactively.

The current process, however ad-hoc, is the process we have at this time and we must follow it until a better process is put in place. Part of this process is a review by the TSC of any proposed REPs; this review does not require that any TSC members be involved in drafting or commenting on the REP itself, and not doing so does not mean that the TSC has not been a part of the process. Neither do TSC members need to be “technically qualified on the topic” or experts. Being able to explain a REP to non-experts is part of the point of the review.

And finally, no one is required to make “active contributions” to a proposed REP to have their comments heard or acted on by the authors. If a proposed REP does not incorporate all requests simply due to available time, then that proposed REP should not yet be accepted.

On TSC membership and its expertise

You seem very unhappy with the way that new members join the TSC. The requirements for a new member joining are open and published in the TSC Charter and in the Project Governance section of the documentation. There are no TSC members who “enter[ed] by the back-door” because there is no “back-door”. The only members who did not go through the current entry process are those founding members who were part of the initial batch of membership or joined shortly thereafter, before a more formal process for joining involving REP 2005 and an incubation period was put in place.

I’m not aware of “10+ startups” who have applied and been rejected stealthily. All membership applications that I have seen come to the TSC membership have involved a prepared document from the applicant, discussion by the TSC (confidential, so opinions may be honestly expressed), and an anonymous vote. The applications are documented in the public minutes. In the past couple of years I can think of one company that was rejected for not meeting the requirements, and one company that went through the same process and was accepted. I do know that one company has been removed recently by a decision of the TSC for lack of attendance, and one other company was at risk of being removed until they clarified their standing.

I also want to make it clear that TSC membership is conferred on organisations, who assign someone from their organisation to represent them on the TSC. Organisations may change their representatives at any time. This happens for a variety of reasons, including internal promotions, leaves of absence, change in role or responsibilities, and resignation.

I wish to also ask you to scale back the ad hominem attacks you are making on the TSC members. Using descriptive phrases like “the corporates” and “backed by X company”, and stating that most of the members of the TSC have “never contributed to a REP, or ROS with relevance” are not helpful, not correct, and are divisive.

Running down the list of the TSC representatives who attended the June TSC meeting (I selected June because July and August meetings typically have lower attendance due to summer vacations):

  • Ralph Lange and Christian Henkel have been involved in ROS activities for years, and lead the groups in Bosch that do make heavy contributions.
  • Jaime Martin Losa’s company produces the FastDDS package on which much of ROS relies.
  • Angelo Corsaro’s company produces the Cyclone DDS package on which much of ROS relies.
  • Chris Lalancette from Intrinsic is the project leader for ROS.
  • Lou Amadio from Microsoft is an engineering leader who has shepherded many important contributions to ROS from his company.
  • Tomoya Fujita from Sony has contributed more to the core of ROS than most of us, and other engineers from his company are likewise significant contributors.
  • Ian McMahon from TRI is a very long-term contributor and represents a heavy user of and contributor to ROS.
  • Henning Kayser and PickNik have been involved in ROS for many, many years and made huge contributions.
  • Jeremie Deray and other engineers from Canonical have contributed a significant portion of the security capabilities of ROS. (The Canonical representative has changed to Mirko Ferrati since the June meeting was held.)
  • Michael Orlov from Apex.AI leads the development of ROSBag2 and other engineers from Apex.AI have been heavy contributors for a number of years.
  • Alberto Soragna and other engineers at iRobot are heavy contributors and have provided much of the current knowledge on executors.
  • Adrian Macnell from Foxglove (a startup) and his engineers (although he’s one, too, but now he has to run a company) provide arguably the best visualisation solution for ROS, contributed the MCAP format and its myriad of client libraries, and have made heavy contributions to ROS.
  • Jerry Towler from DEVCOM is a long-term contributor to ROS.
  • Aaron Blasdel from Amazon has a long history of contributing himself, as do many other engineers from Amazon.
  • Matt Robinson is the current lead of ROS-Industrial.
  • Andrei Kholodnyi and Wind River are some of our leading expert contributors on real time execution.

Although some of the people on that list are now in management positions, I’m not aware of any who are not engineers. I certainly wouldn’t describe any of them as “corporates”.

I think it is also worth explicitly noting that while the majority of the TSC members are “from industry”, that does not make them a separate group from “the community”. The ROS community is made up of a diverse mix of hobbyists, students, and professionals. Giving “the community” an “equal say” to “industry” implies that those members who use ROS for work are somehow separate from the rest of the ROS users. Although TSC members who represent a company are in part representing their company’s point of view, they all do it in a way that is balanced with their views as part of the community and what is best for ROS as a whole.

For a purely non-industry point of view we have the three community representatives. I have seen complaints that some of the community representatives are not active. If you do not agree with the activities of a community representative, the correct course of action is to not vote for them the next time they stand for election. This is one reason why I encourage everyone to think seriously about who should be their community representatives and vote accordingly. The next community representatives election will be starting in September.

Politics and “anonymous lobbying” in the TSC

I’m not aware of any politics or “anonymous lobbying” in the TSC. If you are, I encourage you to provide specific examples so myself and Kat can correct that behaviour.

I also encourage you to clarify your statement that working with the TSC is “complicated”. Broad, generic statements like that do not help us improve processes. Specific problems and examples of them do.

On the ability to contribute to ROS

Can you please be more specific about what policies the TSC is setting that have “changed rapidly in recent years”? I’m not aware of a rapid change in how ROS is run as a project other than the instantiation of the TSC itself.

I’d also like to hear specific examples of it being difficult to contribute to ROS, so we can understand why someone found it difficult to contribute, and try to improve that experience. However I note that the TSC does not gate-keep contributions to ROS other than the REP process, so there shouldn’t be any involvement by the TSC in making new feature or bug-fix contributions to the source code.

On confidentiality and transparency in TSC discussions

The TSC has rules about confidentiality for good reasons. Members must feel free to give their opinions honestly and openly without fear of reprisal in any way in order to encourage free discussion. Confidentiality is rare in the TSC, and typically relates to things such as forthcoming product launches, interpersonal discussions, and potentially contentious votes.

Confidentiality may seem to contradict the desire for transparency, but the TSC has so far been able to balance both. We think the trade-off of allowing confidential discussion to occur when necessary is worth it.

The TSC charter, which you linked to, is clear that any information not given in the public minutes should be considered confidential by all TSC members. It also specifically states:

  • 1 - Statements made by individuals in discussions surrounding a vote or topic of contention which are not reflected in the publicly posted minutes should be considered confidential.
  • 2 - Statements made by individuals regarding companies, research groups, individuals, or other entities for which they are not also members should be considered confidential.
    […]
  • 5 - Affiliates of TSC member organizations or guests of TSC events privy to confidential information will be held to the same standards set out in this section.

If a member of the TSC has been passing on to you confidential discussions and comments from other TSC members, then that TSC member is violating the TSC charter. If the member does not agree with the rules of the TSC, then they should either raise this during a TSC meeting (and comply with the charter until it is changed) or voluntarily leave the TSC. Breaking the rules just because they do not agree with them is not acceptable or professional conduct.

On the “slowdown” in Discourse engagement/involvement

We’re not seeing a slowdown in Discourse engagement and involvement. At worst, it’s stable. On some statistics, it’s growing. This can be seen in the Annual ROS Metrics Report that @Katherine_Scott spends a great deal of time and effort putting together for the community. It is also visible in the inbuilt Discourse statistics, shown below. The following data is monthly; it is not cumulative.

12 Likes

Similarly, a long post responding to the various points above:

See below. Answered with @gbiggs points in more detail. Shortly, most input we got (again, see here) of REP-2014 concerned opinions about what should be in such a REP (and/or how it should be written). You seem to be into writing papers a lot these days so you’ve probably noticed you’ll encounter lots of types of reviewers, some more constructive, some more qualified in the art, but the least you’d expect from all of them is to have writen at least one paper. Otherwise, It’d be infuriating to have them critize the structure of your paper and its content, since they’ve never written one.

You’re implying here that I’m just complaining and pointing fingers. That’s just wrong. See previous comments. If what you need is a summary of my proposals (besides the complaint concerning the rejection of REP-2014, that I believe I’ve made clear) here’s one for you:

  1. Lack of Transparency: I criticized the lack of transparency in the TSC’s decision-making process, particularly in the evaluation of REP-2014, and called for more openness and accountability.
  2. Non-Technical Rejection: I expressed frustration that the feedback for the rejection of REP-2014 was mostly non-technical, despite the proposal feedback being discussed, reviewed, re-iterated based on initial feedback, which we addressed in detail.
  3. Political Nature of TSC: I criticized the TSC for being more political and less technological, and questioned the value of the current TSC setup.
  4. Lack of Strong Contributors: I criticized the TSC for missing strong contributors, key players, and individuals with a strong track record in the community.
  5. Anonymous Lobbying: I expressed concern about anonymous lobbying (mostly corporates, and under confidentiality) and the confidential tone of ROS TSC discussions.
  6. Need for Community-Centric Governance: I called for a change in the current leadership towards a more community-centric governance model.
  7. Inconsistency in Standardization: I criticized the inconsistency in the TSC’s decision to reject standardizing a benchmarking approach in ROS 2, despite existing implementations and the consensus reached among those involved in its development.
  8. Lack of Accountability: I criticized the opacity and lack of accountability in the governance structure, and called for proper mechanisms to replace inactive or non-contributing representatives.
  9. Slowing Down of Innovation: I expressed concern that recent policies and events are slowing down the innovation process in the ROS ecosystem and discouraging many from continuing to contribute.

Good for @Katherine_Scott and I acknowledged this piece already, as for the rest, I disagree. We got no such “feedback with the possibilities of addressing those concerns and re-submit the REP”. The REP has been Rejected and marked as such and as per my understanding of the REP process, there’s no way out now. The message hasn’t been, Víctor and group, please address these concerns and re-submit, the message conveyed as I understand it has been Víctor and group, this REP has no room as REP, so we’ll reject it with this feedback. We encourage you to make a docs page.

We both know in both cases (REPs or docs) can be changed with a simple PR, let’s not over-complicate.

This point has been made very clear. We’ll try and follow what’s being recommended here.

That’s clearly not what happened in here. This is brought out of context. REP-2014 was discussed before the summer and rejected over the summer. It’s unreasonable to expect any prompt reactions to comments dropped in the PR over summer time when it took months and multiple presentations only to get it in the agenda. If any of those comments were of relevance for the decision, the such a decision should’ve postponed with feedback back to the authors.

If what you refer to is my complaint against NVIDIA’s attitude of fork-and-use(but-not-contribute-except-for-marketing) then sure, you’re right and I mean it. We had a similar situation in REP-2008 reviews. TSC requested that we had NVIDIA accepting it as well (phrased as “buy-in from the larger acceleration community”). NVIDIA ignored this whole thing completely (difficulting interoperability across accelerators). The TSC insisted and I had to go ahead and implement it just for the sake of addressing that red line established at the time. The TSC and REPs efforts should favor those willing to invest time on topics of interest, not impose hurdles because one company is not aligning.

Answering this in details would take lots of time. Lots. I’ll try to be short:

There’s plenty of organizations whose applications took many months while for others it happened overnight jumping over the TSC intake process. Not implying these organizations should (or should not) be there, but can you confirm Intrinsic went through the scrutinity of the whole TSC intake process (and note I was present when the hand-raised vote happened for Intrinsic)? Can you confirm Foxglove went through it as well and had all the credentials generally requested concerning REP-2005? Can you confirm (other than the founding members, as you call them) that all the rest went through it?

You also imply you don’t know of various startups applying and being rejected yet I myself did so, multiple times, with multiple companies, over multiple years obtaining creative input. Also discussed this with other frustrated applicants. Not only the information you have seems incomplete, but you point to a link which contains extremely few details about this. Where’s the transparency here? Where’s the associated information with each application? Is this also “confidential” so that the TSC members can express their views without fear?

From an applicant’s perspective the TSC mostly seems an invite-only-club. Not something you can generally apply to.

Feedback is meant constructively and intends to convey the message that most TSC represent big companies and act aligned with that (even at the point of throwing personal attacks). Also, you’re brought this out of context. My exact words were “We need a more diverse TSC committee that encourages and includes more community representatives, at least as many as corporate ones, and who actively engage with the ROS community’s issues and needs”. I mean corporates from the perspective of individuals who run company’s daily operations (and that I believe is its meaning), but not as engaged with the technology.

Don’t get me wrong, I know very well the relevance of those big names/companies and how we all fund the efforts we then invest into ROS but that doesn’t mean I can’t have my own opinion on how things could work better from a community governance perspective and that is precisely what’s being expressed here.

A better TSC structure would have more startups/smaller companies (which unsurprisingly and acknowledged by most, are the ones who contribute the most when they have a TSC sit) and more community representatives. At least as much as “corporates” . Again, not meant negatively. It is what it is, and in my opinion, not what we need the most right now. We need to incentivate contributions strongly as the transition from Intrinsic is happening.

Again, the majority of the inviduals representing these organizations have not participated in a REP, nor are known for their outstanding ROS contributions. You discard these arguments but I ask you again to reconsider and re-evaluate things. Let me touch on a few persons in that list that I’m happy to voice my opinion about. Taken from the official source of TSC representatives https://docs.ros.org/en/rolling/The-ROS2-Project/Governance.html:

  • Christian Henkel (Bosch, let’s not mix things up, if Ralph’s not entitled anymore he should be an observer at best and should have no vote): Don’t know Christian, AFAIK he has not partcipated in REPs. I also don’t know of many contributions from his side neither. :x:
  • Jaime Martin Losa (eProsima): Jaime is a good example of a corporate with no relevant ROS contributions :x:
  • Mirko Ferrati: don’t know of any contributions that justify him being here, neither involvement in REPs. By the way, what you claim about Canonical is incorrect. They have been rotating the TSC members a few rounds in the last few years and have not contributed a significant portion of the security capabilities of ROS. In fact, many of the security issues in ROS 2 remain as of today, many years after reported. As I said before, there’s nobody taking ownership of the real security issues in ROS 2. Without meaning to criticize the work of Jeremie and other Canonical colleagues at the SWG (I know they are pressed by other matters and have very limited resources) contributions security-wise from Canonical have mostly been small utilities and a great marketing use of ROS. :x:
  • Patrick Musau: was selected but didn’t show commitment with the project. Various others in the past requested an avenue to replace him. These requests were answered with a “please vote someone else next time”. Hilarious. :x:
  • Oliver Michel: I saw very few contributions from Oliver and most concerning his own products and campaigns. I don’t know enough to have a strong opinion, so I’ll jut make this observation: It’d be great if TSC members (regardless of whether they are community reps, corporates, startupers, etc.) are encouraged to contribute to those aspects of most relevance for ROS, besides their own products. TSC members are requested a portion of a PM to participate. Let’s make good use of that :wink:

I’ll repeat my words above the current setup of TSC “thought-leaders” (most of which have never contributed to a REP, or ROS with relevance) adds very little value.

Let me explain my thoughts differently: companies nominating TSC representatives should ensure such representatives have a strong ROS track of records by themselves and as contributors (in the various forms that could take, but existing). Why? Because 1) we want direct encouragement for companies to hire ROS robotics engineers (that’s a great reason to participate in this community and encourage people to contribute!), 2) it’s crucial they understand the technical impact of their decisions and repercussions across the codebase, and 3) those TSC members themselves will contribute directly and remain technical, not political.

For example, Sony (Tomoya) is a great example. He’s (as far as I know) one of those who’s walking miles beyond his solutions/software/products to improve the overall ecosystem. He’s not only sending PRs across the ROS stack, he’s reviewing REPs and from what I’m seeing here/life, also trying to constructively collect feedback. We need more Tomoyas and less managers in that list (since corporates seems rejected).

I’ll paste in here again my comment above implying that the overuse of confidentiality together with the current TSC which is mostly represented by big companies (and corporates) is leading to “confidencial lobbying” or “anonymous lobbying”

Please see:

Contributing to ROS is not just sending a PR (as we’ve all acknowledged repeatedly). It’s also about coordinating a subgroup of the community, pushing forward a new consensus to avoid reinventing the wheel (aka REP), organizing meetings, etc.

I have experienced plenty of difficulties with the ROS 2 Hardware Acceleration Working Group. Soon after running the group for a while successfully we got changed into a “community” category, followed by changes in calendars and events, followed by its complete removal from the official list of ROS 2, followed by removal of (access) rights to events we had previously created, etc. This has been a pain. Received plenty of complaints about it from members of the group and they were right. It became hard to track things after so many changes. You claim this is not TSCs domain but I recall reading in the TSC notes about some of these changes so that’s why I made that assumption. Whoever’s responsible for this, it’d be great to take it into account.

I’ll be brief in here and just say that I don’t agree with the argumentation nor think confidentiality is rare based on my own experiences. I believe TSC members should be transparent with their own votes and opinions, and help community members achieve their goals, not present obstacles. I believe the current setup favours lobbying and I’d encourage OSRF to consider recording TSC meetings and disclosing them.

Good thing then! You own the data @gbiggs and I don’t believe to have access to the information you point to. If you say so, I can believe it. Happy to be wrong here. The following is what I’ve had in mind for the last year approximately and what I’ve based myself on to make my statements:

(source: https://metrics.ros.org/, see changes in the slope of committers, see slope of Discourse posters)

(source: 2022 ROS Metrics Report)

I’m merely stating the fact that while you make certain claims about people from the TSC saying certain things to you privately, these claims are not supported by anything besides your claim. I respect that you may want to protect your sources, but if this is to be taken seriously, we need to hear it from more than just one person.

That is your view. I stand by my position that everything that is really relevant is discussed in the working groups in an open and transparent fashion and that the TSC is primarily alignment, and I can only re-iterate that the only datapoint you’ve brought so far is the decision regarding the REP proposal.

You’re claiming that I do this in bad faith. Have you considered that I might be doing this because I’m fed up with one man making himself out to be the victim here, and being blind to the valid criticisms of his proposal and genuine attempts to bring the matter forward on a technical level.

3 Likes