True, a number of components seem better designed, and software engineering practices are used more from the beginning. A problem when porting a complex system built on top is just that the application code, often silently, relies on properties of the communication layers (think for example about when exactly to ask tf for transforms and what to consider beforehand). If such behavior changes (even if it changes to the better), problems will appear.
That’s probably the crux of it; how many of these implicit assumptions will surface, and how long will they take to find and fix. I guess that’s an argument for both never moving to ROS2 (so we don’t have to deal with it) and moving to ROS2 immediately (so we don’t write any more code that relies on ROS1 idiosyncrasies). :-/
I’m not sure. To be honest, I haven’t had time to use ROS2 yet, so I cannot say how similar the APIs are, and therefore how viable it is to write such a script. (BTW, the reason I haven’t used ROS2 yet is because it’s still missing some features (actions, nav) that we require for our paid projects, so that ruled out ROS2 without even trying, even when starting a new project.)
In the past, I found conversion scripts pretty useful for small upstream API changes that a lot of dependent projects have to do. Some examples are here:
My hunch is that the ROS1 -> ROS2 conversion is more than just a few renames, so it’s hard to write a script that gets it right (but I could be wrong here). More importantly, when starting to switch over to ROS2, I would be willing to invest some effort into learning ROS2, and after that doing the conversion manually probably wouldn’t be more effort than running an imperfect script and fixing it up afterwards; especially since doing the renames is only 10% of the effort, the other 90% is debugging the subtle changes in behavior that @moritz mentioned. In contrast, any effort put into learning the pluginlib conversions is lost, because you only need that knowledge once. But if it’s easy to do, why not write a conversion script and see how well it works.
BTW, here is an example that’s probably closer to what a ROS1 -> ROS2 script would look like:
It’s a collection of scripts that was used to switch from the old rosbuild system to the new catkin system 5 years ago. I’ve never found it terribly useful for the reasons I outlined above: I had to learn catkin anyway, and the scripts didn’t get it 100% right, so it was easier for me to start off with a clean catkin template and manually move stuff over. It’s mostly mechanical work, but you end up with a clean and correct result.
I’ve had second thoughts about if we really need a shim. In the end, what I want is to mix ROS1 and ROS2 packages in the same system. If the ros1_bridge works well enough, maybe that’s all I need (I haven’t tried it yet).
My reasoning goes like this: What’s great about ROS1 is the wealth of community-contributed packages. In recent years, I was forced to use a different robotics framework on some projects. The workflow was usually like this:
- week 1: figure out what components we need for this project, discover that 90% already exist as ROS packages
- month 1-4: port all those ROS packages over to Different Robotics Framework
- month 5-6: actually implement the missing 10% from scratch
I often banged my head screaming “It would be so easy if we just used ROS”, and I would very much like to avoid repeating this experience, where
Different Robotics Framework == ROS2. In the foreseeable future, this problem isn’t going to go away, because there are tons of ROS1 packages out there that are still useful, but no longer actively developed, so there is little chance of them being converted to ROS2 any time soon. If the ros1_bridge works perfectly, that would allow us to use the wealth of packages from ROS1, while porting everything over to ROS2 piecemeal. I’ll have to invest some time trying the ros1_bridge soon.
In the end, I think that the question of limited resources might be the most powerful one. If
Open Robotics has N hours to work on ROS, how many should they spend on ROS1 and
how many on ROS2. If we need M hours of work to make ROS2 viable, then the math is
simple; more time on ROS1 means a longer time until ROS2 is ready.
That simple math was true from the first day of work on ROS2 four years ago. It is not an argument for or against any strategy.
How about this question: If under ideal circumstances, ROS2 could be ‘feature-complete, production-quality, viable as a complete replacement of ROS1’ in X years. And if providing further ROS1 releases added Y years to that. Who would actually be hurt by those additional Y years of delay? Amazon? Will they go bust over a delay of ROS2? Microsoft? IBM? Intel? What’s the worst that could happen, and to whom, for additional delays to that state of ROS2?
We can see clear and obvious harm in even talking about stopping ROS1 releases (though not harm to the financial sponsors of Open Robotics), but I fail to see any harm in delaying a “really ready” ROS2 further. The feature-complete DDS robotic middlewares that ROS2 is based on are available to anyone desperate enough, no team in the world is blocked without ROS2 becoming “really ready”.
So if Open Robotics wants to be a foundation serving the whole open robotics community (not just their financial sponsors), what is the morally best decision?
I’d like to echo that point: the underlying package code is open, as is the build farm code. There are multiple organizations that run their own copies of the build farm now, usually to produce custom distros for internal use. So if Open Robotics were to stop doing ROS 1 releases, anyone else can pick it up. You can even do it now. I’m of course not encouraging the creation of forks, which I think that most of us will agree would be bad, but it’s not crazy to have a backup plan if you’re inclined to worry that we’ll abruptly walk away.
Of course, as you also point out, you probably don’t have funding to support that kind of effort. But then, neither do we! Amazon is now generously funding our use of AWS resources to host our build farms, but nobody is paying us to spend time releasing or even maintaining ROS 1. For reference, from our internal staffing plans I estimate that preparing and releasing a ROS 1 distro requires 8-9 person-months of effort from our team. Since leaving Willow Garage in 2012, we’ve received approximately $0 directed at maintenance or improvement of ROS 1. And even if we had such funding, we’d still be limited by the number of people on our team. If you’re currently trying to hire software engineers, especially in the bay area, you can likely sympathize.
I’m not complaining. We’ve made it work because we believe that it’s important. But the trade-offs laid out by @mkhansen and @wdsmart are real: time that we spend on ROS 1 is time that that we’re not spending on ROS 2, thereby (further) delaying the development of the latter. And the risk in that delay is also real: at some point organizations that eagerly want ROS 2 because ROS 1 doesn’t meet their needs will decide to stop waiting and instead build or buy something else, likely a proprietary solution.
Planning future ROS 1 distribution(s)
For reference, from our internal staffing plans I estimate that preparing and releasing a ROS 1 distro requires 8-9 person-months of effort from our team.
Since leaving Willow Garage in 2012, we’ve received approximately $0 directed at maintenance or improvement of ROS 1.
So for each future release of ROS1, how much $ would you want to make more releases? Just enough for 8/9 person-months every year? Can it be raised via kickstarter?
If the problem is having developers, what other options exist? Does the releaser have to be in the bay area? Can it be a freelancer / outside organisation similar to what I guess is still happening with ROS answers?
time that we spend on ROS 1 is time that that we’re not spending on ROS 2, thereby (further) delaying the development of the latter. And the risk in that delay is also real: at some point organizations that eagerly want ROS 2 because ROS 1 doesn’t meet their needs will decide to stop waiting and instead build or buy something else, likely a proprietary solution.
Let’s check this risk. We assume there is a company X that will wait for ROS2 if no more ROS1 releases are done (8-9 person-months gained every year), but not wait if ROS1 continues to have releases. So company X has at least 3 options:
A: continue waiting for ROS2
B: build or buy something else
C: Provide a dev-seat equivalent of 8-9 person-months every 2 years
It seems to me that both A and C will always remain more economically viable than B. So how realistic is scenario B as a risk, really, if the crucial difference can be made by just dropping ROS1 releases?
A: continue waiting for ROS2
B: build or buy something else
C: Provide a dev-seat equivalent of 8-9 person-months every 2 years
Just throwing my perspective here, having worked for a variety of robotics startups over the last few years. Option A is not an option - If ROS1 can’t be used for it’s fundamental deficiencies, then they either need ROS2 or an alternative to ROS. Option C is not an easy one either - most developers who actually write code do not have control over the purse strings - it’s not always easy to convince management to opt for C when that money could just as easily be allocated to buy a license or support contract for alternate solutions so that they have a chance of delivering a product now.
I am one of those roboticists eagerly waiting for ROS2 to mature. I still have to deliver to my employer - so if ROS2 is delayed, I will go with option B. It’s a no brainer, really.
To give a sense of scale for 9 person-months in the Bay area, here are some numbers that I’m going to make up. I don’t know how much Open Robotics pays it’s people, but I’m guessing you can’t get a good software engineer for less than about $150,000 a year in the Bay Area. My rule of thumb (as someone who hires people and students at a university) is that people cost about twice their salary, once you add in benefits, insurance, and all the other stuff you have to pay for. So, $300,000. I don’t know what Open Robotics’s overhead rate is (overhead is the money you have to dedicate to keeping the lights on, and stuff like that), but here at OSU, it’s 53%. So, if I spend $100, I have to give and additional $53 to the University. This is probably high for a business, but I really have no idea. Call if 50% for easy math. So, your employee costs $450,000 a year. 9 months of this is $337,000. Assume that I’m way off on all my calculations, and cut that in half (I think that this is unrealistically low, but let’s do it anyway). That’s $160,000 a year, which is a heavy lift on Kickstarter.
Asking Open Robotics to make the “morally best decision” is a passive aggressive way of claiming that they’re choosing to do the morally wrong thing. Passive aggressively suggesting that there has been embezzlement at Open Robotics is just an ad hominem attack (at worst) or simply shows an ignorance of how companies operate (at best). Neither helps your argument.
From my point of view, as someone who has to raise money in a similar way to try to do similar things (albeit at a university in a research context), the explanation @gerkey gave makes sense. It’s not the optimal situation, and it sucks, but that’s the way it is. Unfortunately, the days of Willow Garage pumping millions of dollars into ROS development from a magical pot of money are gone.
Of course, all this arguing doesn’t help solve the problem.
Jumping ship and burning bridges has never been a great recipe when you are dragging a lot of weight (community) with you.
@bmagyar why do you talk about burning bridges? Who ever mentioned this? It is a bit sad to see this being brought up out of the blue. This post and similar events are exactly to not do that and get the feedback from the community and prepare it for the transition.
If you already have an inflight or working ROS1 application it can, and probably should stay on ROS1 until that application has exhausted its useful lifecycle. Any new or next generation of that application should be targeted to be built on ROS2 and getting the libraries there to support it.
Using ROS 1 and ROS 2 side by side using https://github.com/ros2/ros1_bridge works perfectly as we have detailed in our ROSCon talk: https://roscon.ros.org/2018/presentations/ROSCon2018_ROS2onAutonomousDrivingVehicles.pdf.
So a) you do not need to do transition over the night and b) you could even keep running your system as ROS 1 and ROS 2 hybrid.
@tkruse noone is stating that Melodic will be the last release. We are polling the community and will decide based on the feedback.
I’d like to take a brief moment to note that this thread and all of the valid concerns discussed herein seems to have stemmed from a relatively innocuous discussion had in the ROS2 TSC, and that neither those meeting minutes nor @gerkey’s reply are advocating for any near-term deprecation at all.
In general, I think that these are good conversations to have for planning purposes, but I’d like to request that people don’t project the positions taken in this thread by those who are not Open Robotics management as positions which Open Robotics is in fact standing behind or advocating.
I’d also like to remind members of the ROS2 TSC of their roles as well - the ROS2 TSC does not guide the entirety of Open Robotics efforts any more than it guides the efforts of the entire ROS community, so statements like ‘[The ROS2 TSC] are polling the community and will decide [on the ROS1 release plan] based on the feedback’ is a bit of an overreach.
If you have been cleanly separating your ROS 1 code per 5Cs and want to end up with the same workarounds in your application code as Moritz mentions above, I think that scripting can actually take you really far.
If you want to make your product really right, then your hunch is correct. At the very end ROS 1 has some fundamental and software engineering issues that logistics and home robotics companies have been able to work around but developers in automotive, medical and aero space were not, because the requirements in the latter 3 domains are too stringent. ROS 2 however is suitable for the use in these domains.
I will try to explain in couple of points why I think so:
- ROS 1 lacks quality of service (QoS) settings which ROS 2 now has. This is such a fundamental feature for especially multirobot setups on corporate WiFis that we had to do tons of work on top of ROS 1 sub/pub to mimic them. When porting to ROS 2, developers will actually have to sit down and think about how they want their communication between nodes, robots to look like. After they will think about it, the implementation will be super straight forward but still not search-replace like.
- ROS 1 lacks managed nodes, lifecycle manager and heartbeat functionality. Again, we spent tons of work in the past to mimic this behaviour but at the best ended up with the hack. And you can not build a production system without these features. ROS 2 has rclcpp_lifecyle (managed node API), launch system that understands and can command managed nodes and will also have heartbeat. Here again, the design of a ROS 2 system will have to change a bit, e.g. you will have to think what to do in the init, configure and run state and when to go to and what to do in the failure state. However after you will figure this out you will be able to develop systems that are repeatable in behaviour and much more robust.
- Security. I see that developers do not take this serious at all. But we will see robots be hacked in the future and there will be bad press for those robots where you can just simply do the following in your root shell “rostopic pub /control TwistCmd 5m/s”. ROS 2 offers security but again you have to consider which parts of your ROS system you wanna run as trusted and how much of overhead the security brings.
- Large data support. I hear from many parties that TCPROS is not fast enough for them. And so many went ahead and implemented their own SHM transport in ROS 1 or used a different framework because ROS 1 does not provide SHM.
At least 2 DDS providers (DDS is a middleware of choice in ROS 2) offer their own SHM transport.
- Real-time and determinism. Below 3 talks go into details why making your system be real-time with ROS 1 is a hack. ROS 2 offers constructs such as separation of Nodes and Executors, Waitsets (in works), https://en.cppreference.com/w/cpp/thread/timed_mutex, … which will allow for a synchronous flow of data but which also means that as well in this case you need to sit down and re-think your architecture.
In general I absolutely see the concern of a large code base being built on top of ROS 1 and I think that we should start carefully and openly planning which ROS packages are the most needed and get them ported to ROS 2 and give the community enough time to make a switch. For starters I think that these 2 lists offer a pretty good overview and I would suggest adding to them:
If this is not an acceptable solution, then the very next thing that comes to my mind is that we go and try to find money for additional OSRF people that will let them do continuous ROS 1 releases without impacting any ROS 2 work. Just my personal opinion but I think that it would also make sense that this money comes from parties that largely depend on ROS 1.
I think you may still need at least one more ROS1 release on an LTS Ubuntu distro – here’s why:
- ROS2 isn’t fully featured yet – as an example: actions will hopefully be part of the upcoming release, but I can imagine it taking 1-2 release cycles at a minimum to get those fully ironed out.
- Until those features are there, some large portion of the developer and code base will not end up on ROS2 (I can imagine that the trajectory will be very similar to ROS1 – a few folk outside the core team end up using the C-release, but the majority of the community shows up for D- or E-releases). Even if they start to use ROS2, they’ll probably be bridging things in from ROS1.
- If we suggest that the E release will be the first very highly useable release, then I think E ends up supporting Ubuntu 18.04 (since it is tail end of 2019?)
- However, ROS2 releases after that will probably end up supporting Ubuntu 20.04 and not 18.04 – meaning that you no longer have ROS1 packages to bridge over to your ROS2 installation. Right?
To me, that says that after April 2020 (only about 18 months away), it gets a lot harder to run the current ROS2 release with any ROS1 packages bridged in. Is that logic right? By releasing ROS1 on Ubuntu 20.04, you can an extra 2 years of bridge availability.
Planning future ROS 1 distribution(s)
Asking Open Robotics to make the “morally best decision” is a passive aggressive way of claiming that they’re choosing to do the morally wrong thing.
Let me expand. As a non-profit open-source foundation creating software without commercial license, Open Robotics has 2 distinct groups of stakeholders: The sponsors and the users. The obligations towards the sponsors are legal and economical. What obligatios are there towards the users? I call those obligations towards the users “moral”, making choices impacting the users “morally wrong” or “morally right”. I am open to other wording, such as “contradicting the self-defined mission of the foundation”, or “counterproductive in establishing and maintaining trust”, but I suggest using “morally right/wrong” for the sake of brevity, if nothing else.
In solving the problem of deciding whether to have more ROS1 distributions or not, I hope this clarifies the values and goals of Open Robotics towards the non-sponsoring users, as a driving factor for the decision to make at hand.
suggesting that there has been embezzlement at Open Robotics
My post with the given statement has been censored btw. I was trying to say that when talking about the funding of ROS1 releases, it is not helpul to say that in the past Open Robotics has has spent (6 releases) x (337,000) ~= 2 million , but has received 0$ for that purpose. Because there is a gap of 2 million dollars in that calculation.
In solving the problem of deciding whether to have more ROS1 distributions or not, I hope this explains what additional information is required to explain the economical decision making.
@mikeferguson - that’s a very good point that bridging needs to be supported for a longer time. I would think that the Ubuntu 20.04 ROS2 LTS release could also support 18.04, just as the Bouncy release today supports both 18.04 and 16.04. I think that’s a reasonable request, maybe @gerkey can comment on whether that could be done to help with the transition.
All your points are spot on. As a commercial company just joining the “ROS” wave, we see absolutely no reason to do anything with ROS1. ROS2 on the other hand is very attractive for many of the reason that you point out.
FWI, I can’t speak for other commercial companies, but if we do what we think we are going todo, we are likely to spend about $1B on robot software development over the next 10 years. $0 will go to ROS1. Much of it is likely to go towards ROS2 functionality. That’s not immoral, its just good business. We don’t see ROS1 as a “commercially” viable technology base, at least not for our markets (which are personal defense and security robots.)
If ROS2 weren’t a thing, we’d likely spend our money on enhancing our own proprietary system and then releasing as much of it as we can as open source code.
I don’t believe that we are alone. ROS2 can become the foundation of machines serving a wide variety of commercial market segments that are each worth 10’s of billions of dollars. As a result, if the community can provide the courage, vision and know how to lead, then large sums of investment are likely to flow into ROS2 over the next couple of years.
OK, so that’s a coherent, evidence-based argument that I think I buy. That does kind-of set the clock running, though. Question: how much of ROS1 will we end up porting? Bonus points: How long will this take? Can we, as a community, reasonably do it over one LTS cycle?
At the end of the day. Development will go where money requires it to. There are many business leaders in this thread who have knowledge of this information of where they are allocating their resources towards. I think if one goes through and reads through the responses, in most cases it is clear whether or not they are interested in putting resources towards ROS1 willingly (or because of due diligence).
For an effort this large, its probably best to write some scripts that reasonably translates some code patterns into ROS2 code patterns. Either that, or provide a shim at a clever spot. Either that, or a giant blog post/cheat sheet from an expert who has done a ROS1->ROS2 transition of their code base.
Does anyone know of an existing ROS2 robot that has been transitioned from ROS1? I would be up to analyze some code and see if there is a potential static translation/best effort translation I could write a script for for most ROS nodes.
What I potentially see in this is a service (yes, $$) that helps businesses port their code from ROS1->ROS2 in a timely manner. Not everyone needs to go through this manually for all parts. For such a large community and a large open source code base - automation here is a winning strategy for all.
@gerkey @rgariepy @mikeferguson @wdsmart @gavanderhoorn Is anyone on your teams doing something like this? Are you guys allocating resources towards it? How many of you would utilise something like this?