Splitting the Autoware.AI repository and changing the organisation

[wjwwood]

Just a few clarifications from me, which I’ll do inline, but in general I’d say that splitting things up in repositories can be helpful, especially for consumers of the project (assuming it’s a framework or SDK, rather than a stand alone application). However, it definitely comes at a cost, and I would buy the argument that it may not be the best use of resources depending on your schedule. On the other hand, that argument can be used to justify an indefinite amount of technical debt, so I always hesitate to follow it blindly.

So on balance, I don’t have a recommendation for your group, just comments

@Dejan_Pangercic

• It is difficult to guarantee a separation between safety-critical and non-safety-critical code.

This is a) certainly not applicable to Autoware.AI and b) whether a package is safety-critical or not could also be asserted via a cmake macro and depending on that you can build which CI rules and checks apply to that particular package or not.

My impression for why this was the case is that reviewing code for the two cases is wildly different and so you need to look at each pull/merge request to first determine how you should review the code (to what standard) and you have to make sure during iteration that you don’t “leak” into more critical parts of the code. With separate repositories it is clear, if you need to change the safety-critical code it will require a separate pull request and that is easily noticed and audited.

@Dejan_Pangercic

you do not need a feature and code freeze when doing releases, these can both be one

Perhaps I miss the point, but as @gbiggs mentioned, I don’t think this has any impact on code freeze versus feature freeze.

@Dejan_Pangercic

much better traceability from feature request, design document, code, tests, PR and CI. E.g. OSRF folks have to manually link github issues with PRs and PRs with CI/CD, e.g. https://github.com/ros2/rclcpp/pull/639#issuecomment-470160609 . Here I am not 100% certain if this just a gitlab feature but in gitlab in the monolithic repo this happens automatically

I think you’re mistaken here. GitHub has the same features as Gitlab w.r.t. reporting the status of CI/CD. We do not use this feature because we have custom infrastructure and have not taken the time to use the GitHub API to automatically report the build status. We do have this in ROS 1 and ROS 2, but we don’t use it as much in ROS 2 yet, e.g. here’s a pr picked at random:

Fix rendering order for movable text by roehling · Pull Request #1292 · ros-visualization/rviz · GitHub (look at the “All checks have passed” section or for the green check marks next to the commits)

@Dejan_Pangercic

easier user/organization management

As others pointed out, I think it’s actually easier to be granular with permissions (a good thing imo) if you use multiple repositories within an organization.

@Dejan_Pangercic

as @davetcoleman mentioned this is a limitation of bloom

This is frustrating for me because I have explained this so many times

It is not a limitation of bloom, but instead a limitation in our process or maybe a limitation of distributed VCS (i.e. git) depending on how you look at it. Basically it boils down to the requirement that each release has a tag, which I think is a reasonable and good thing to have for people consuming your software, and is reinforced by things like GitHub tying your tags to releases (not every tag is a release but every release is a tag).

If you keep that requirement then you cannot (in my opinion) realistically tag the repository in such a way that more than one version of a package can be represented easily. For example, if your repository has foo and bar packages in it, and on your first release you release both at version 1.0.0 and so you can use the tag 1.0.0 for the release. But then you want to release foo and not bar, so you set the version of foo to 1.1.0 and keep bar at 1.0.0, so but then what tag do you use? foo-1.1.0?

What if later I update foo and bar to 2.0.0, do I then use 2.0.0 again or use two tags, one for each? If the user wants to get the latest version of the software that works together, which tag do they choose? Remember that bar-1.1.0 could be newer than foo-2.0.0. Also bar-1.1.0 might not have a released version of foo as its peer, but instead some in between versions state.

I could go on for a while, but the point is that it’s a conceptual issue, not a limitation of bloom.

@dirk-thomas

Of course you could tweak bloom to support releasing subsets of packages from a single repo. But realistically is anyone interested to put the effort into doing that?

You can already do this using the ignore files. It would be trivial to have this as a command line option to bloom.

However, the conceptual issue still stands, if you find a small bug after a big release that you fix upstream then you still need to do a new release and then you’re back to the tagging issue I mentioned above. Even if you tag new releases for all packages and only want to actually bloom one package’s new version, then you could do that but you’d have a mismatch between binaries (debian packages) and what’s in the source tree, which is confusing for contributors and consumers of the software.

The right answer, if that’s your concern, is to split the repository up. Otherwise, I think you need to live with the “useless version increments” in unchanged files and the slow release process and the redundant updating of basically unchanged binaries that you mentioned.