Splitting the Autoware.AI repository and changing the organisation

@gbiggs It seems planning might have Control according to the discussion #1719. I think we need some consensus to rename top level diagram.

@shinpei0208 @aohsato @Dejan_Pangercic Are there any comments?

Then the first thing we should do is come to a consensus on what constitutes planning and what constitutes control.

In my experience in the manipulation world, once you have a set of joint states over time (a trajectory) to achieve, anything after that is control. Creating that trajectory is planning. My experience in mobile robotics has planning as producing a path to follow to the goal and a shorter path to follow in the immediate vicinity of the robot, and deciding what velocities to drive and turn at to follow that path being control. I think the two are fairly similar. But I donā€™t know if autonomous driving follows the same convention or not.

1 Like

@gbiggs I misunderstood your proposal. Mission Planning (and maybe Actuation) in Autoware is similar to Control you mention. I think both ideas make sense once we can get consensus :slight_smile:

Which repository should each vehicle interface such as ymc and as packages be in in your proposal? Drivers repository for interacting with vehicle hardware?

@kfunaoka I think any third-party driver/binary should be in the drivers repository, so that we can avoid future problems when a new binary is needed

If itā€™s a driver for specific hardware, especially something we can eventually graduate out of Autoware (e.g have the hardware maker maintain it) then it should go in drivers.

There is some debate about the naming of core_control going on, and also about the separation between control and planning and actuation. Speak up if you have an opinion.

I know Iā€™m a bit late to the party on this one, but I wanted to put in a couple of items regarding the repo naming. I agree with @Dejan_Pangercic that core_localization is separate from core_perception. Localization is the process of determining your estimated position in a reference coordinate frame - in our case, using inputs from multiple sensors. The best definition I can give for perception is the processing of raw input data gathered from environmental sensors into usable, logical representations of those data (which is why I, personally, would include the drivers under perception). For example, I would say that the NovAtel driver and any post-processing on individual pieces of data would be under ā€œperceptionā€ but the combination of data from multiple input sources and the comparison of the estimated position output from those combined data against a reference coordinate frame would be considered ā€œlocalization.ā€ Similarly, if we were doing camera-based localization, the camera driver and any neural networks which process the raw images into representations of real-world objects would be ā€œperceptionā€ while the comparison of the images against a known set of images for the purposes of estimating position would be ā€œlocalization.ā€ At their core, I think ā€œperceptionā€ is about representing objects outside the vehicle and ā€œlocalizationā€ is about representing the position of the vehicle relative to the outside.

Regarding core_decision_making vs core_planning, I think itā€™s a little less clear. My opinion would be that planning pretty much covers the path-planning and decision-making processes but Iā€™m not as solid on this one.

1 Like

@JWhitleyWork thanks, this is a great explanation.

Regarding

Control (algorithms) are separate from planning (algorithms), so @gbiggs had it right.

Regarding separation of decision_maker and planing packages maybe this definition helps:

  1. Global planning (=decision_maker) => on demand service, interfaces to apriori map and traffic information,
  2. Behaviour planning => open problem, Darpa urban challenge: rule-based, other companies: search-based method, our favorite: robust POMDP
  3. Local planning => some deterministic sampling-based methods (GMT) or form of MPC, pre-recorded path, RRT

As you see 1 is on-demand, non-realtime package that can interface to many different sources of data. 2 and 3 are online, realtime algorithms with a minimal set of dependencies. Hence the split.

Not too late. We can still change!

I feel that localisation and detection could be classed as both perception if you take perception as being ā€œunderstanding the state of the worldā€. Iā€™m not religiously devoted to dividing things up like this, though. Itā€™s more that it gives us one repository instead of two or three. Nevertheless, functionality and binary releases may make us consider splitting them into two.

We want the drivers separate because:

  • the idea is that any new drivers will eventually get pushed upstream, so the drivers repository should always be aiming to be empty.
  • Drivers should get relatively stable which means we donā€™t have to make releases of it very often, whereas perception we would like to see a good cadence of new features and algorithms.
  • Drivers are almost certainly going to be safety-critical software, whereas much of perception might not.

I feel the same way on this one. However we recently had a long discussion here about where to draw the lines between planning, control and actuation. There was no easy place to put it. In general I think that the breakdown @Dejan_Pangercic gave is the right one from both the interfaces point of view and in terms of thinking of things as stages.

I think it is important to remember that repositories are in many senses a release unit, rather than a strict delineation of ā€œthese must be used togetherā€ or ā€œthese form a single application/pipelineā€.

1 Like

I agree with @Dejan_Pangercic and @JWhitleyWork that core_localization is separate because perception is a too big category.
What do you think of dividing perception into localization and object_recognition?

In my studies, ā€œlocalizationā€ == ā€œego perceptionā€ and was considered a subset of perception for the same reason @gbiggs described. The overall goal is to perceive the state of the environment, your ego position within that environment is just another aspect of the state of the environment.

Either way we decide Iā€™m fine with, just wanted to provide some additional experience!

After moving the packages to the new repositories, I can make a couple of observations:

  • There would be almost nothing in core_control, only one or two packages.
  • core_perception would be huge.

Based on these observations, I think it is worth considering merging core_control and core_planning, and splitting localisation out from core_perception to try and get that repository down to a more manageable size.

utilities is also huge, so there is some thinking that needs to be done there. Some of the packages, such as the one providing build flags, are used everywhere as well which is a nasty dependency to have.

drivers is full of stuff we can get rid of, like unnecessary forks of existing ROS packages.

1 Like