Current state of web and mobile monitoring and teleoperation

ROS Projects
1

Hi, I have been working on a DIY mobile robot project (initially ROS1 and have moved to ROS2) and was looking into my options for remote monitoring and teleoperation from an Android tablet.

I first came across ROS-Mobile which seems like a great project, but is currently ROS1 only (and fair enough). I could probably bridge it but it would be nice not to have to.

Next I thought to myself “there’s probably a common web interface people use for this stuff, I can just run that”, but even then there did not seem to be a “plug and play” solution. The Robot Web Tools Github has an actively updated JS library which is promising, but most of the demos and tools on their website seem to be outdated and/or broken.

There seem to be implementations here and there (such as Jack Pien’s Hadabot) but nothing common. The best thing I’ve found is WebViz by Cruise (demo here), which I will have a go at but seems quite heavy and lacks some of the features I’m interested in.

My questions then is, is there a standard/simple ROS web interface for monitoring and/or teleop, and if not is that something the community would find valuable?

I imagine it would be useful not only for phone/tablet connections, but also to connect from Windows laptops etc. And from there it could be wrapped in an app for ease of use on the various platforms. With sufficient features it could even be an alternative to RVIZ and rqt.

For my purposes the following features would be useful (but I’m sure the community has varying needs):

  • Topic echo
  • Simulated Joystick
  • Grid map
  • Camera
  • 2D (or 3D) TF View
  • Gamepad passthrough (if possible, to use controllers such as this)

I assume such a thing would be a built on the RobotWebTools server and client libraries.

If this doesn’t exist I’d be keen to have a crack! My web programming experience is very limited (background is more C++/embedded systems and MATLAB/control) but I’d give it a go.

Please let me know your thoughts (and my apologies if this is not the right place to post this).

Webviz Example

https://user-images.githubusercontent.com/177461/88328232-fa27b880-ccdc-11ea-9167-0d9339254729.gif(image larger than 5072KB)

6 Likes
2

Josh,

I couldn’t agree more with you that these tools should exist, and share your surprise about the fact that they don’t! I’ve spent several years at Savioke leading a team that built web components like the ones you describe (plus many more), and in talking with several other companies found out that there is a lot of overlap between the components we needed and built and the web components other companies needed and built as well.

We actually have a blog post coming out where we are proposing a full-stack platform to address this misfit and reduce duplicate work by enabling better sharing. I’ll cross-link it here once it’s public and will be looking forward to your feedback!

6 Likes
3

Allow me to add

  • (dynamic reconfigure) parameter change

to the list of features.

3 Likes
4

Do you mean something a little like this:

This is running off a webserver on the robot. As such anything with a web-browser can run it. (Please note that this is an older version and we’ve added many new features since then such as multi-waypoint route plans etc.)

2 Likes
5

A couple things that might be relevant to this discussion.

I would second that it would be great to see a concerted effort around building a suite of browser based UI tools for ROS.

3 Likes
6

As promised, here is a link to the (announcement of the) blog post we wrote on the topic:

Feedback welcome!

3 Likes
7

Howdy - this is a great writeup Christian. There’s only one piece I don’t agree with…

…and as a result, the above capabilities do not exist as commercial-off-the-shelf or open-source modules that robotics companies can use.

In fact, we (Formant) have almost all of what you’ve talked about here already generalized and run in production with many thousands of systems and hundreds of different morphologies around the world. I think you may have actually worked with our head of engineering (Abraham Dauhajre) at Savioke.

As far as we know, there’s also at least Rocos, Freedom Robotics, and inOrbit working on this problem with great-looking capabilities as well.

At a high level, we’re slicing the problem slightly differently than you’re suggesting and focusing exclusively on the data & remote operations infrastructure. Autonomous capabilities (like SLAM & navigation) and application-specific capabilities (like shelf scanning & traffic light detection) are not easily generalized across domain and morphology, and those are typically the pieces that our customers’ or OEM partners’ core competencies are based around.

Slicing the problem that way also makes it quite clear that it’s also our responsibility to go much deeper than you’ve described into data management features like on-demand ingestion, in-app analytics, and collaborative capabilities like timeline annotations and integration with systems like Jira / Zendesk / Slack / etc. You correctly identified that embeddable UX components, real-time APIs, white labeling, customer portals, and other “construction kit”-type capabilities would be a big deal - which it took us until a quarter or so ago to catch up to.

I do wish there were better open source capabilities here, but it’s quite a lot of infrastructure to run, and critically important to the companies relying on it. We do at least try to make it easy on startups by using a consumption-based pricing model, so cost is always easy to control.

I also really like your package management concept. In my experience, there is currently a very anemic ecosystem of robot companies buying individual software modules from each other. We’re working on introducing embeddable apps for interface customizations now, but that’s kind of a different thing.

Here’s a simple observability share link to check out some of our stuff if you’re interested.

1 Like
8

Jumping right into this discussion to draw the attention on a WIP project that we have at eProsima: visual-ROS, which we already advertised on these shores back in November (you can check it out here).

In a nutshell, this engine is meant to provide a user-friendly web-based graphical IDE for programming with ROS 2. The implementation is based on Node-RED, an open-source browser-based graphical editor allowing low-code programming of event-driven applications. Node-RED allows to link together customizable nodes to generate flows, which are deployed to the runtime and stored as JSON files. To leverage this tool, we’re implementing a library in charge of converting these flows into WebSocket APIs, which will be then translated to ROS 2 language via SOSS.

visual-ROS-architecutre
visual-ROS-architecutre1378×479 34.8 KB

A dedicated team is already hands-on, and we hope to be the carriers of very good news ASAP!

4 Likes