@bshin-lge we looked at this extensively couple of months back and we also talked to lots of simulation companies and none could really full-fill what we were asking for.
In short these would be our prioritized requests:
- Simulation runs can be scripted
- Simulation runs can be parallelized
- Lidar + Camera Sensor models out of the box (including interference as Geoff points out)
- Simulation can happen at faster than real time (and in headless mode as Geoff points out)
- Traffic participants have behavior models
- (Pseudo-)random traffic scene generation. Possibly there is a connection to some database with the pre-configured traffic scenario, e.g. https://www.pegasusprojekt.de/files/tmpl/pdf/AVT%20Symposium%202017%20Database%20traffic%20scenarios_Folien.pdf.
The entire, non-prioritized list would be as follows:
- Computer hardware requirements for simulation should be listed
- Interfaces (mocks) supported by simulation (CAN, ethernet, …)
- Crisply defined software APIs
- Support/integration for ROS. Here I would like to point to this discussion https://github.com/CPFL/Autoware/issues/1724#issuecomment-443079012 that we already had with LG guys and I felt that our opinion was not really taken seriously.
- Fidelity of sensor models
- E.g. do you accurately model different surface reflectivity/absorptivity, incidence angle, etc…
- How to create our own sensor models?
- Map formats supported
- Maps that come with it?
- How to include our own maps?
- Precision/ real-time capabilities in general
- Use for stress testing, e.g. an extremely high sensor load (>10 cameras, >10 lidars, etc) in realtime, precision/ real-time capabilities/guarantees during stress testing
- Can simulation run in faster than real time?
- How is the behavior of other cars governed in simulation?
- Are there API’s/plugins that we can define to modify this behavior
- How are traffic scenarios generated?
- Is there any randomness in the simulations?
- If yes, can we specify sources of randomness?
- Is there collision detection?
- Can simulation run in a headless state?
- E.g. Can it run without a GUI?
- Can we programmatically access the simulation state?
- Or do we have to interact via filesystem (e.g. xml)
- Are car dynamics simulated?
- E.g. suspension, slip etc…
- Can the simulator be integrated with the CI runner and does it output simulation/test results in e.g. junit format for test result validation?