Integration of ROS2 with Machine Learning End-to-End Platforms

fkromer · 2020-04-09T19:19:24.916Z

I was wondering if there’s integration of ROS2 with E2E ML platforms. Can you guys give me some hints?

gbiggs · 2020-04-09T23:40:07.622Z

Possibly a good question to ask at ROS Answers.

Katherine_Scott · 2020-04-10T00:26:38.102Z

Maybe someone on the AWS RoboMaker team like @Aaron_Blasdel, @Cam_Buscaron, @emersonknapp can work together with the AWS SageMaker team to put together a tutorial. I know this is something I have heard others request this in the past. I know I would love to see it.

fkromer · 2020-04-11T09:48:46.472Z

I searched for questions + answers but did not find any. I posted a question myself now: https://answers.ros.org/question/349109/ros2-integration-into-end-to-end-machine-learning-platforms/

fkromer · 2020-04-11T09:52:57.708Z

@facontidavide For the data visualization part it could probably be interesting to integrate PlotJuggler into ML platforms. Most ML platforms provide a Python based plugin system, a Python SDKs (some JAVA SDKs and Go SDKs as well, C++ SDKs are very rare) and RESTful APIs for integration. Due to the fact that PlotJuggler is based on qt integration wouldn’t be straight forward though.

fkromer · 2020-04-11T11:18:29.140Z

That would be great of course. In many cases for early phase startups however it won’t be possible to use AWS right from the start due to costs. Also keeping data on on-premise hardware may be critical. Integrations with platforms offering open source variants which allow experimentation with on-premise hardware (like e.g. cnvrg.io CORE, KNIME Analytics Platform, H2O.ai) would be more valuable for a broader potential user group.

facontidavide · 2020-04-12T12:05:37.191Z

Quick feedback. PlotJuggler is and will remain a stand alone desktop application. But there is no reason why it shouldn’t be able to visualize information from another running application, through inter-process communication.

Isn’t this what ROS is good at ?

fkromer · 2020-04-12T13:22:03.999Z

First it was not obvious to me that plotjuggler ist qt based. Despite of the typical qt look it could have been a web app. Web apps can be integrated more easily. I totally agree.

Katherine_Scott · 2020-04-13T16:21:37.701Z

I guess man, that certainly hasn’t been my experience in the bay area. What I’ve seen in my travels is that most start-ups are trying to minimize the number of boxes they have on prem. From what I’ve seen spinning up a cloud instance (doesn’t really matter which one) has a lower barrier to entry and better repeatability.

fkromer · 2020-04-14T09:15:25.725Z

What I’ve seen in my travels is that most start-ups are trying to minimize the number of boxes they have on prem. From what I’ve seen spinning up a cloud instance (doesn’t really matter which one) has a lower barrier to entry …

I fully agree. However data privacy regulations in Germany are the issue here.

facontidavide · 2020-04-14T11:32:06.856Z

If you think that writing:

  sudo apt install my_application

represent an entry barrier compared with using a web-app that require an internet connection all the time… well, I can not help you

Anyway, @fkromer, if you want to discuss this potential integration with PlotJuggler, I will be happy to see if I can help implementing it.

fkromer · 2020-04-15T18:05:33.094Z

facontidavide:

If you think that writing:
  sudo apt install my_application
represent an entry barrier compared with using a web-app that require an internet connection all the time… well, I can not help you

On Ubuntu in my personal dev setup I’m totally fine with apt or snap. However when it comes to MLOps where you potentially have to plumb a lot of stuff together there is no way around dockerized software. It’s the defacto standard. Of course there are exceptions to this general best practices e.g. when production code runtime matters (e.g. like when checking and optimizing the prediction time of classifiers on ARM boards, etc.). However obviously the majority of people is not interested in checking and optimizing these kind of things. Otherwise there would be already tooling available w.r.t. this.

facontidavide:

Anyway, @fkromer, if you want to discuss this potential integration with PlotJuggler, I will be happy to see if I can help implementing it.

Please do not misinterpret my request. I’d never ask for help without giving back either (a) contribution to open source or (b) money for your development effort. I have everything I need right now. And in case I need something I will implement it on my own

However I saw that you search for sponsors. The effort for setting up and maining MLOps functionality is drastically underestimated. It’s way more efficient to use the generic functionality of a E2E ML Platform and integrate domain, use case, whatever specific stuff on top. And I think there could be huge demand for ROS2 integration with E2E ML Platforms including w.r.t. visualization in the future.

milLII3yway · 2021-03-24T17:04:43.022Z

@fkromer 11 months later… have you found any solution for the ROS2 integration with E2E ML Platforms?

fkromer · 2021-03-25T09:55:18.029Z

@milLII3yway We are not using ROS2 cause it does not give us benefits in comparison to other technologies ATM. Means no progress w.r.t. platform integration at our side.

LewisLiuPub · 2021-03-29T05:29:04.711Z

Sounds OpenVINO toolkit is not included here.
You can develop applications and solutions that uses deep learning intelligence with the Intel® Distribution of OpenVINO™ toolkit. Based on convolutional neural networks (CNN), the toolkit extends workloads across Intel® hardware (including accelerators) and maximizes performance.

The OpenVINO toolkit has already been wrapped to ROS2 and ROS.