How to import my overhead grid map to Nav Stack

[hosh]

Intro to project
We are working on developing a mobile robot that has to carry packages around in a warehouse. The thing really matters in this project, is that robot should avoid explorations as much as possible. Due to the limited field of view of the on-board sensors, we have to put a few overhead cameras on the warehouse ceiling, looking downward. These overhead cameras are being used to segment the floor to the occupied/free pixels, leading to an occupancy grid map (we call it an overhead grid map). So we have an overhead grid map and a grid map which is obtained from onboard sensors (the onboard grid map is a subset of the overhead grid map, and the grids in overlap will fuse together).

What we have
We have generated a dataset for overhead image segmentation in Unity and developed a ROS2 package for a CNN based image segmentation. Another ROS2 package is implemented to do Global Planning based on the NF2 algorithm. The result global path will be fed to ROS2 Nav Stack.

The Question:
As you may have noticed, the packages are decoupled from Nav Stack. We are interested to develop these packages somehow to can be used as a plugin to Nav Stack. Our main concern is about developing the overhead grid map. My first thinking was developing the overhead grid map as a costmap 2d plugin.

Develop a costmap 2d plugin that receives the segmented images, and the output is a costmap layer. However, to the best of my knowledge, it may violate the costmap 2d plugins’ rules. For example, the overhead plugin has to resize the layered_costmap_, which seems just static_layer is allowed to do.

Another one is developing a package that subscribes to the SLAM toolbox and expands its grid map by importing an overhead grid map.

What is the best way to merge this package to Nav Stack?

[smac]

This is a good question and I see no reason why this shouldn’t be possible relatively easily. If you’re starting with some kind of occupancy grid information, there’s really no difference between that and a map generated by SLAM.

(by the way, NavFn uses a navigation function for planning, may just want to use that, its wicked fast managing its own memory buffers)

An aside, if I were you as the system designer, I would probably place an intermediate node between the occupancy cameras and the costmap to consolidate them into a single occupancy grid that then navigation uses. You don’t have to, strictly speaking, you could have N of your ceiling_occ_layers, but for practical efficiency and designing a system that can more easily scale as N grows, you may want to consider having a single ceiling_occ_layer given from a node that combines them.

@hosh

My first thinking was developing the overhead grid map as a costmap 2d plugin.

That’s exactly what I would start with as well.

@hosh

However, to the best of my knowledge, it may violate the costmap 2d plugins’ rules. For example, the overhead plugin has to resize the layered_costmap_ , which seems just static_layer is allowed to do.

I’m not sure what these “rules” you speak of are, but remember that plugins are designed to be modular and they can do literally anything you want. There’s nothing special about the static_layer, in fact what you’re trying to do is directly analog to the static_layer in trying to provide a floorplan to the navigation system to size the building / provide information about the static infrastructure / even maybe include current state information about moving things. So if you want to resize layered_costmap_ like the static_layer does, power to you.

[hosh]

Thanks for the comprehensive explanation.

@smac

An aside, if I were you as the system designer, I would probably place an intermediate node between the occupancy cameras and the costmap to consolidate them into a single occupancy grid that then navigation uses. You don’t have to, strictly speaking, you could have N of your ceiling_occ_layer s, but for practical efficiency and designing a system that can more easily scale as N grows, you may want to consider having a single ceiling_occ_layer given from a node that combines them.

Exactly it would be the case. Its image segmentation package’s task to stitch images from different overhead cameras and outputs one global image.

@smac

I’m not sure what these “rules” you speak of are

I also not quite sure

[hosh]

The following is a short update of what I did on this project until now:

1. Generating synthetics dataset for ceiling image segmentation in Unity (Cite to Patric)
   

2. Developing a ceiling_segmentation ROS2 package using TF 2.4 -Including Unit and DenseNet- that segments the images to the Free VS Occluded pixels - (Link to GitHub repo soon)
   

3. Developing a ceiling_perception ROS2 package that accepts the segmented images and generates an occupancy grid map - alpha version

4. Developing a ceiling_perception_plugin ROS2 package as a plugin to the costmap layer - alpha version
   

   
   

TODO:

• For now, we assumed ceiling_camera poses and robot pose are known. In the next version, we will develop a method for localizing the robot w.r.t ceiling_cameras (unknown robot pose)

• Although an accurate global map is not needed we may develop an elevation map (2.5 D) using RGB-D ceiling cameras instead of current 2D map

Any thought and suggestion will be appreciated.