I’d be a bit more restrictive regarding versioning. All the core packages must have the same MAJOR.MINOR version to indicate that they are only compatible within each other, but PATCH releases can evolve independently. Basically follow Semvers:
I’d add a simulation repository that may contain any simulator-specific code, scenery and models. However, code in this repository is also a candidate to be pushed upstream to the respective simulator (e.g. LGSVL, Carla, etc.)