| ROS Resources: ROS Homepage | Media and Trademarks | Documentation | ROS Index | How to Get Help | Q&A Help Site | Discussion Forum | Service Status |
Hello ROS Community! I am excited to introduce plans for an open-source node graph tool for robotics! It will be similar to the visual programming tools commonly used as development aids in many professional video game engines, except tailored to robotics.
To gauge interest and make sure we cater to your needs, we are asking that you take five to ten minutes to complete the survey linked below. More information about the project can be found in the survey introduction.
Thank you!
This looks similar to behavior trees; what is proposed benefit here over existing behavior tree packages?
Great initiative!
The blueprint in Unreal are really great for fast iterations and high level logic.
It could be a great new asset for ROS developers.
Great question @swan
One of the biggest advantages of node graphs over behavior trees is that they are more intuitive. Node graph logic more closely mirrors how we think through a sequence of steps, which makes them quicker and easier to design. This also makes it easier to understand the big picture when reviewing a graph, which is useful for when modifying or debugging a graph. I intend for this to expedite development for experienced developers, and also lower the barrier to entry for those new to robotics or with less programming experience.
A couple other advantages include a node graph’s ability to better represent states (e.g. fault states, behavioral states) and to run parallel processes (though ROS certainly helps with this in behavior trees).
All that being said, behavior trees have their own strengths, and the “best” tool often depends on the specific use case. I can even envision scenarios where a combination of both would be ideal. For example, I can imagine a node graph node that executes behavior trees as being something useful.
That might be so, but are they equally expressive? It’s easier to be more intuitive, e.g., a language that only allows sequences of actions is very easy to understand, but the question is whether that comes at the price of not being able to express certain things (e.g., in my example, loops). Another way of asking this is: can any BT be translated into a node graphs, and/or can any node graph be translated into a BT?
The next question to ask is: for practical applications, how complex would a node graph become if it had to represent something that is fairly compact in BTs, e.g., fallback nodes. Note that this requires variables and lists.
We experimented with using graphical programming (extending Blockly) at a mid-sized robotics company I worked at in the past (~100 robots), and our experience was that very quickly, as the complexity of the algorithm increased, they provided no benefit over regular programming languages. The graphs became so big that they became hard to navigate and grok. Another finding was that data structures, a key enabler for reducing program complexity, are often poorly supported in graphical programming frameworks.
Thank you for your thoughts @chfritz. This is exactly the type of feedback we are looking for.
Our goal is for this tool to be capable of representing just about anything that a behavior tree can represent. In fact, if anyone can think of something a behavior tree can represent that a node graph couldn’t, I’d be interested in hearing about it.
I should mention that a key feature that will enable this level of representation and hopefully address some of the concerns you laid out is what I am currently calling a control wrapper node. These are nodes that wrap one or more nodes or sub-graphs and execute these nodes or sub-graphs in some some way that is defined by underlying code or by a more complex underlying graph. So a simple example would be a control wrapper that repeats the enclosed node a given number of times. Another example would be a “Do Until” or “Do While” control wrapper that executes something until a condition is made true or untrue (analogous to reactive sequences or reactive fallbacks in behavior trees).
We are hoping scenarios like this can be mitigated with the ability to encapsulate logic into sub-graphs. In the example image I included in my original post, each blue node represents a sub-graph that could be expanded to reveal more logic (in the way the PickObject sub-graph is expanded in that example). If this graph were to be represented without sub-graphs, it would be hard to manage, hard to understand, there would be tons of repeated logic everywhere, and the number of variables in the scope of the graph would get out of hand.
I’ll be honest that I have not given data structures much thought, but you bring up a great point. I wonder if this is another area in which control wrappers could help. For example, a control wrapper could be defined to take in a data structure as an input and then call the enclosed block of code for each element in the data structure. What other examples do you have in mind in which one might want to use data structures at this high of level?
Unfortunately, @chfritz is right here. Visual programming is inherently limited and it is no match for regular programming languages, both in expressiveness and ecosystem. The idea is not new and has been tested many times in many contexts. It always ended the same way. For example, UML as a programming language:
As a visual framework, you are bound by a 2D plane to express things and relations. One mathematical fact in this field is that the number of planar graphs is much less than the relations you can express using unrestricted graphs. If I remember correctly, we do not have this restriction for 3D spaces. So that’s only bad news for Flatland. You may say those are theoretical bounds that never happened in practice, but the decades of practice also show the same.
This is the problem of all code generation approaches. It is very easy to start, but eventually, you need to include conditions, loops, data structures, and classes, as well as set up your compiler infrastructure, linters, version control, and all other parts of a full-fledged development environment.
This is a very good idea!
So that ROS can be used more in the world of autonomous mobile robots that respect the industrial standards that make the automotive or aeronautical world robust
For this, the tool would have to be close to what can be done with Simulink (from Mathworks / Matlab). Each block would have to embed a C++ or Python code that is only a calculation algorithm that can be tested independently (algo development, unit tests, debugging, validation, etc.). The block itself only passes the input and output variables and the possible parameters. The best would be for the blocks to be able to be clocked on top with a scheduler!
For maximum readability, it would be necessary to be able to group several blocks in a marco block, to have very macroscopic views, as well as very detailed when you enter each of the sub-blocks, with several floors. And of course a browser to navigate between the different levels.
We can only strongly encourage you to make this type of tool that ROS really lacks to switch from the world of IT developer to engineer and open the world of robotics to other scientific cultures that will be very beneficial to it.
To clarify, this tool is not intended to replace traditional programming languages, but rather to complement them by fulfilling at least one specific purpose, similar to the purpose that FSMs and behavior trees fulfill. This intended purpose is to abstract complex sequences of behaviors, especially those involving parallel processes or conditional execution paths based on the outcomes of previous tasks. We tend to get lost in such sequences when limited to traditional text-based programming because it is hard to visualize the sequence as a whole.
But to your point, there are limitations to visual programming, so I definitely agree that the detailed algorithms of each discrete behavior should still be implemented using traditional programming languages. Similarly, the larger program that encapsulates these behavior sequences would likely be best handled in a traditional programming environment.
I just realized that I posted an old revision of the survey that did not include the page that recruited support for the project. If you are willing to contribute to this project but weren’t able to indicate such because you tool the old version of the survey, please DM me!
Thats Cool. Makes it Easier to Visualize the nodes
Hey, nice tool!! I will check It out 
