I’m Alan G. Sanchez, a robotics Ph.D. student at Oregon State University. I am currently interning at Hello Robot, working alongside Amazon’s AWS RoboMaker team on a project I want to share with you all.
Below is a gif demonstrating our robot broadcasting its camera feed to a WebRTC stream and viewed through the media playback viewer in the AWS console.
Here is a quick overview of these two AWS features:
KVS is a fully managed streaming service designed to securely ingest and store, video, audio, and other time-encoded data from connected devices to AWS. KVS lets users playback video for on-demand viewing and quickly build applications that utilize computer vision and video analytics.
KVS with WebRTC is a service that enables real-time communication (RTC) across browsers and mobile applications via simple APIs. It uses peering techniques for real-time data exchange between connected peers and provides low latency media streaming required for human-to-human interaction.
Additionally, KVS can be utilized to connect and monitor a robot fleet remotely from your AWS console. We’ve developed a guide for those interested in integrating these cloud extensions to their robot. Check out our open-source tutorial on GitHub.
Please view our Hello Robot forum post, which includes our acknowledgements for those who helped get this project up and running.
I’m excited for the ROS community to check out my guide and these cool cloud extensions. Please share if you do.
Thanks for sharing Alan! We were looking at doing something similar for an outdoor forklift project last year. Looks like the stack has matured a bit more.
Nice work! There seems to be some latency in the video stream of around three seconds. Any idea where that is coming from? In my experience, latency of webrtc video streaming is usually just around 200ms. I looked at your scripts but couldn’t find anything obvious. Is this perhaps due to the way Kinesis is encoding the video?
Hey Chris,
Are you using the similar workflow with rostopics → RTSP stream → gstreamer sample with kvs webrtc?
One area where i saw latency creep up is the rostopic to RTSP stream conversion. You can double check this by verifying the data in Rviz vs opening a VLC player on the local machine with the data and opening the RTSP stream. It might need some tuning in setting up the parameters right to reduce latency.
The other option is to directly send the frames from rostopics using the SDK instead of conversion to RTSP and using gstreamer. The approach in the demo is easiest to setup but going directly to frames and using the SDK will give you better control and performance, at the expense of more dev effort. Let us know how it goes!
Gstreamer is multithreaded so crashes and errors are bound to popup, you might have to rerun the pipeline multiple times to get your deisred result (in general), so latency there is not unexpected. You can try the queue ! element in the pipeline if you haven’t already.
You can also use ffplay (from FFmpeg) or gst-play-1.0 to test your pipeline
