RQT in ROS2

Hello ROS community!

We are excited to announce that PickNik is working with Amazon to port the plugin-based graphical user interface RQT to ROS2 for the December Crystal release. The RQT packages provide an easier entry-point for a broader user base than command line usage. RQT is built using the open source cross-platform framework QT.

Our primary approach to this effort will be:

  • Target only the RQT-common plugins and dependencies
  • Add support of Python3 and ament build system
  • Support OSX and Windows
  • Rework ROS middleware plugins to work with DDS
  • Support the other core ROS libraries that are currently being developed by being first adopters of their new APIs, such as rosbag, roslaunch, and actionlib.

We invite the community to provide feedback and suggestions on RQT by replying to this thread. While this is an ideal time to break API, given time and resource constraints, enhancements will be secondary to porting existing functionality. However, this thread will also act as a rough roadmap to guide future enhancements.

We believe this is an excellent opportunity to contribute to the ROS2 development effort and a first step in preparing PickNik to migrate MoveIt! to ROS2. We are grateful to the community and corporate partners like Amazon that help drive improvements for ROS.

Thanks,
The PickNik Team

15 Likes

This is my rqt wishlist, in an order that is a rough combination of increasing difficulty and decreasing priority:

  • rqt_image_view - Iā€™d like to see this before much anything else.
  • rqt_plot - maybe just choose or implement from scratch a simple single plot type rather than integrating multiple ones, make it work well enough, donā€™t consume tons of cpu with default settings.
  • rqt dynamic reconfigure with an interface more like rqt_image_view- Iā€™ve made my own ros1 version of this: https://github.com/lucasw/dynamic_reconfigure_tools/tree/master/rqt_dr_single (it has some issues with reloading and servers not responding I need to clear up) - instead of a tree of all possible servers there is just one that is saved as a setting into the perspective file, but it can be switched with a drop down menu. Iā€™ll make a ros2 version myself given ros2 dynamic reconfigure (?).
  • message publishing and echoing, service calling and action calling could also use similar treatment to be more like rqt_image_view, and with the configuration hidden be presentable to an end user who knows nothing about ros. Save configuration to the perspective file so it would be possible to set up a widget in a ui that is meant to do something specific on a single pre-configured topic or service as opposed to a tool to explore all possible topics and services etc. on a system in a big tree or list.
  • Individual crashed or unresponsive plugins ought to not bring down the entire rqt instance.
  • rviz-in-rqt - I know rviz2 work is already underway but it would be fantastic if the whole of it could be a small number of rqt plugins. A render core would publish the view/s as Images and be configured through standard ros2 interfaces. rqt_image_view would need to send all mouse events rather than just left mouse clicks. The side panels could be special rqt plugins, or some functions could be generic service caller or dynamic reconfigure plugins and corresponding servers within the render module. Eliminate redundant plugins for rviz like the image viewer (though the one that under/overlays rendered objects is still useful).
  • Convert a .perspective file to a webserver + html5/js, or an android app.

Please do not mistake porting effort with arbitrary feature development. The main goal is to port the existing framework and several plugins to ROS 2 (Python 3 as well as Windows).

If you would like to specific features please feel free to contribute them - either to the current ROS 1 branches or to the -to-be-created ROS 2 branch. A couple of brief notes to your bullets:

There is no reason to implement ā€œyet anotherā€ plotting plugin. There is already (at least) two: rqt_plot as well as rqt_multiplot

Dynamic reconfigure conceptionally / intentionally doesnā€™t exist in ROS 2. The ROS 2 parameters are node specific and replace the ROS 1 parameters as well as dynamic reconfigure.

In the past Qt offered the ability to embed widgets from separate processes (at least on X11). That is not available in Qt 5 anymore and therefore this way of isolating plugins in rqt isnā€™t possible anymore. If you see another way to achieve this please feel free to propose it.

There is already a rviz plugin for rqt (it has some severe issues though). That could be ported as is. Publishing and rendering the rviz views and relaying the mouse / keyboard interaction sounds like a big no-go to me.

This sounds like an incredible complex task - even ignoring that there are no html5/js or Android components atm which could be the target of any mapping.

Please do not mistake porting effort with arbitrary feature development. The main goal is to port the existing framework and several plugins to ROS 2 (Python 3 as well as Windows).

Iā€™m responding to the ā€˜While this is an ideal time to break API, given time and resource constraints, enhancements will be secondary to porting existing functionality. However, this thread will also act as a rough roadmap to guide future enhancementsā€™ above. If there were decisions to be made in immediate porting effort that would benefit the sorts of features outlined above then that is the substance to take away. For instance, if only a subset of the common plugins where to make the cut for the targeted release, the first few items are what I would like to see- and others will have an opinion, and maybe some shouldnā€™t be ported at all? (Iā€™m drawing inspiration from other ros2 threads about if a given package or part of a package isnā€™t used much, or has been superseded, or just is way less useful than some other parts or packages, donā€™t bother porting right now or ever)

There is no reason to implement ā€œyet anotherā€ plotting plugin. There is already (at least) two: rqt_plot as well as rqt_multiplot

If it were quicker to do it from scratch, or just narrow support to a single back end, and that would make the difference between having rqt_plot by the desired time or not, and/or it would end up with better performance, then that would be the reason to consider not just a direct port. (Maybe it should publish the plot as an Image, or a generic vector image message type if pixels seem clunky? Maybe every visual plugin should publish an Image instead of drawing directly to a window- that would simplify several of them, to the point where they are no longer even rqt plugins, just regular ros2 nodes with parameters set through a generic ros2 param setting rqt plugin? If there are strong reasons not to that ties into the rviz-rqt idea)

Dynamic reconfigure conceptionally / intentionally doesnā€™t exist in ROS 2. The ROS 2 parameters are node specific and replace the ROS 1 parameters as well as dynamic reconfigure.

Instead of direct porting there would be an equivalently functional rqt ros2 plugin for viewing and changing per node parameters. (Being able to hide some parameters and save that into the .perspective would be useful if every node has a lot of boiler plate parameters.)

In the past Qt offered the ability to embed widgets from separate processes (at least on X11). That is not available in Qt 5 anymore and therefore this way of isolating plugins in rqt isnā€™t possible anymore. If you see another way to achieve this please feel free to propose it.

Thatā€™s unfortunate. In the much longer term maybe the qt in rqt should be something else (which could also benefit the web/android/whatever cross platform capability, or some of the other items here).

Publishing and rendering the rviz views and relaying the mouse / keyboard interaction sounds like a big no-go to me.

Iā€™d be curious to know why- if it has anything to do with underlying limitations of ros or qt it will help me guide my future pull-request efforts and comments (if it is just timeframe this is something for the longer term, e.g. ā€˜rviz2.5ā€™ or ā€˜rviz3ā€™). Iā€™ve done similar things on a limited scale in ros1 with satisfactory results, I assume ros2 will perform better.

Convert a .perspective file to a webserver + html5/js, or an android app

This sounds like an incredible complex task - even ignoring that there are no html5/js or Android components atm which could be the target of any mapping.

Yes that is true, but again I throw it in here in case someone else knows something can contribute, or in some decision between x or y it turned out x could be more easily converted to those platforms. The short term would probably be a few of those components- a generic node parameter configuration widget, an image viewer, plotter, echo, publish, etc. and just a document to guide how to manually assemble them into something similar to the rqt file, and automation later.

If you would like to specific features please feel free to contribute them - either to the current ROS 1 branches or to the -to-be-created ROS 2 branch.

Of course, Iā€™ve always done this in the past. Iā€™m hopeful that if the right minimum coverage of standard ros interfaces have good general ui plugins then writing ui code now will pay off in my (or someone else in my place) not having it to write much or any application specific ui code in the future, it will just be a matter of selecting and positioning and configuring the widgets (weā€™d be writing ros nodes with ros interfaces mainly meant to be part of a ui that is a thin layer of generic plugins, but that is an improvement over having to write qt).

Do we actually need two separate ways to build graphical user interfaces in ROS2? There is Rqt, which provides dockable widgets, and RViz, which provides dockable widgets and also 3D visualization.
I donā€™t know why ROS1 has two separate GUI widget plugin architectures, but porting GUI widgets from ROS1 to ROS2 is a good opportunity to reconsider this. Is there a reason ROS2 needs to have two GUI widget plugin systems?

I consider RViz as the more useful GUI to visualize a robotā€™s state and control it, and I think that ROS2 should establish RViz plugins as the default way to compose GUIs. I do not see the advantage of having two plugin systems for widgets. In some cases (e.g. image view), it even creates multiple versions of the same visualization functionality.

The only remaining issue I see with RViz is that the 3D view in RViz is not dockable at the moment, i.e. it is always visible and cannot be removed. I think developer time is better spent by making the 3D view dockable and porting Rqt widgets directly to RViz(2).

2 Likes

I actually donā€™t agree. The docking system in rviz is minimalistic and used primarily with things related to rvizā€™s activities. rqtā€™s docking system is far more sophisticated and necessary for some of the features it supports.

One important difference is that rviz panels share state with each other though C++ interfaces whereas rqt plugins can either be c++ or Python and exchange (almost) all information via ROS topics, services, and actions.

I actually see them having different goals, if anything were to be elided, Iā€™d say it should be that rvizā€™s application code (docking and extra widgets) could be replaced with rqt, such that the rviz panels (i.e. the render window, panels, and tool bars) would become rqt plugins and the ā€œrvizā€ application would simply be a specially crafted rqt layout.

Iā€™m actually not sure itā€™s worth doing this, because of the aforementioned shared state between rviz parts which would require either rebuilding those interfaces as ROS interfaces (unnecessary and would have more overhead) or changing rqt so that different plugins could communicate with each other directly (a difficult issue to be sure, given you cannot assume the existence of other plugins nor that theyā€™re c++ or Python).

Any update on rqt for ROS2?

Hello Lalit,

We ported most of the rqt common plugins to ROS2 for the Crystal release. Please see the tutorial page for more information.

https://index.ros.org/doc/ros2/Tutorials/RQt-Overview-Usage/

Stephen

2 Likes

Thanks Stephen, I will check it out.

If anyone wants to help, I plan to port PlotJuggler to ROS2. Any help from people with experience in DDS/ROS2 is welcome.

Is there any plan to port rqt_multiplot?

There is no reason to implement ā€œyet anotherā€ plotting plugin. There is already (at least) two: rqt_plot as well as rqt_multiplot

If it were quicker to do it from scratch, or just narrow support to a single back end, and that would make the difference between having rqt_plot by the desired time or not, and/or it would end up with better performance, then that would be the reason to consider not just a direct port.

This is the only thing I have to say about the latter comment :wink:

image

5 Likes

Is anyone porting* the rest of the rqt plugins? In particular, is anyone porting rqt_graph?

(*Edit, fixed)

rqt_graph has been ported and released!

1 Like

@mlautman - Where? Can you point me to it?

1 Like

is any one porting rviz plugin for rqt?

1 Like

To install rqt-graph. @mkhansen, @Rajesh_E

# sudo apt-get install ros-$DISTRO-rqt-graph
$ sudo apt-get install ros-crystal-rqt-graph

I tried on Ubuntu 18, ROS2 Crystal.

1 Like

@ harderthan- I am working on Ubuntu 18.04, ROS2 Dashing
I have already installed rqt-graph what you have mentioned.

rqt is listing only ā€œImage Viewā€ and ā€œPlotā€ options under ā€œplugins/Visualizationā€

Do I need to install any other plugins to get rviz plugin for rqt?

Iā€™m not sure anyone has taken on the port of the rqt_rviz plugin. When we ported the RQT plugins to ROS2, this one was specifically avoided due to the large amount of work involved and the timeline we were working under. Alberto from iRobot also looked into it, but Iā€™m not sure if they were able to make any progress on it. Itā€™s likely a large undertaking and would require some dedicated effort from knowledgeable individuals.

Related thread:

https://discourse.ros.org/t/porting-the-rqt-rviz-plugin-to-ros2/10309

Stephen