ROS Resources: Documentation | Support | Discussion Forum | Service Status | Q&A

Simulation software requirements

A few things off the top of my head that I think we need from one or more simulators. I say “one or more” because it’s likely that different simulators will be better at different things.

  • Deterministic execution. When executing deterministically, the same starting conditions should give exactly the same results.
  • Ability to control the simulation step (pause, step forwards a specific time, step forwards one loop of the control algorithms, run faster than 1:1 time, etc.).
  • Headless mode for running as part of a CI pipeline.
  • Modular framework (links to the above requirement, but broader). We need to be able to do things like plug in different pedestrian simulation algorithms, different sensor simulation models (even for the same sensor types), and so on.
  • Fault simulation in all parts of the car, from wheels to engine to sensors.
  • Sensor noise and interference simulation.
  • Simulation of different times of day and different weather conditions, including their impact on road conditions and thus the response of the car to control.

@bshin-lge we looked at this extensively couple of months back and we also talked to lots of simulation companies and none could really full-fill what we were asking for.

In short these would be our prioritized requests:

The entire, non-prioritized list would be as follows:

  • Computer hardware requirements for simulation should be listed
  • Interfaces (mocks) supported by simulation (CAN, ethernet, …)
  • Crisply defined software APIs
  • Support/integration for ROS. Here I would like to point to this discussion that we already had with LG guys and I felt that our opinion was not really taken seriously.
  • Fidelity of sensor models
    • E.g. do you accurately model different surface reflectivity/absorptivity, incidence angle, etc…
  • How to create our own sensor models?
  • Map formats supported
  • Maps that come with it?
  • How to include our own maps?
  • Precision/ real-time capabilities in general
  • Use for stress testing, e.g. an extremely high sensor load (>10 cameras, >10 lidars, etc) in realtime, precision/ real-time capabilities/guarantees during stress testing
  • Can simulation run in faster than real time?
  • How is the behavior of other cars governed in simulation?
    • Are there API’s/plugins that we can define to modify this behavior
  • How are traffic scenarios generated?
  • Is there any randomness in the simulations?
    • If yes, can we specify sources of randomness?
  • Is there collision detection?
  • Can simulation run in a headless state?
    • E.g. Can it run without a GUI?
  • Can we programmatically access the simulation state?
    • Or do we have to interact via filesystem (e.g. xml)
  • Are car dynamics simulated?
    • E.g. suspension, slip etc…
  • Can the simulator be integrated with the CI runner and does it output simulation/test results in e.g. junit format for test result validation?

Headless mode, logger (if the accident happens in the simulator, the logger logs this situation) and scenario mode is necessary for running simulation in CI.
I want to use simulator for CI.

CARLA hits several of the requirements that we need.

Here’s another one for when simulating light-based sensors: simulating things like sun glare behind a traffic light or a bright reflection from a glass pane into a camera for an instant.

These are all really good points. What are the specific use cases of simulators as it pertains to people working on Autoware right now? As with @Dejan_Pangercic’s post, I think it’s helpful from our perspective (the LGSVL Simulator team) at least to think about prioritization of features from users’ point of view. We know about some of the use cases of the Autoware Foundation, but it would be interesting to hear from any additional active Autoware developers and what their needs are for their specific use case. As @gbiggs entioned, maybe it will help more clearly define different simulators’ roles and their differences as well.

As for LGSVL Simulator specifically, I’ve addressed some of the feature requests above as they relate to our roadmap.

Currently, our immediate priorities are:

  • API implementation
  • New map generation/import

Some requests (hardware specification description, which exact sensor models are supported) are a documentation need, and we will continuously add to the documentation (

Control/scripting capability of simulator:
We are currently working on implementing a Python API to run the simulator - this includes start/stop, run-time configuration, stepping through/running faster than real-time, controlling the environment, controlling ego/non-ego agents, state/sensor data retrieval.
This will let us more easily define or use standardized scenario descriptions.

LGSVL Simulator is based on Unity, and to my knowledge the PhysX Engine in Unity does guarantee the same simulation result when all inputs are the same. This is certainly a whole topic itself as to the degree of determinism that is required for use cases - the act of running on different machines (if in cloud, for example) can change behavior, and the extent to which it does and whether it’s acceptable is probably an important question.

Headless mode for CI:
We do plan to enable running the simulator in non-rendering mode.

Parallelizing simulation runs:
This is on our roadmap (we need to upgrade Unity version and implement multi-threading)

Behavior models of non-ego agents:
@Dejan_Pangercic Do you mean dynamics models of agents as well or higher-level behavior?
This can be enabled by the API (agents can have specified behavior such as following waypoints or randomness).
Currently by default, traffic vehicles follow the vector map, and pedestrians walk random paths.

Scenario database:
We are currently working on being able to support scenarios, and after this definitely plan to look at generating test suites of scenarios, integrating with existing scenario databases, etc.

@bshin-lge I’d like a plan for simulation support and usage to be put together. Every simulator has its strengths and weaknesses, and that is not going to change no matter how much one simulator tries. It’s not just a question of features but also of underlying technologies used, how features are implemented, etc. So we need to understand what each simulator is best at and where it can be best applied. For example, if I were making a demonstration video, I would probably use LG’s simulator. If I were doing simulated cameras, Cognata’s simulator produces images that are more photorealistic. If I were doing statistical data gathering of planning performance across a range of scenarios, I would also probably use Cognata’s because they have a good system for generating scenarios procedurally and deterministically, and executing them deterministically. LG’s simulator and Carla appear to have better physics simulation and so are good for control testing. Things like this need to be considered and each simulator developer needs to decide on what sort of usage they wants to focus on. Then we can decide how to apply each one.


@bshin-lge thanks for the reply first. To reply to above question first, it would make sense to have non-ego agents behave lawfully as well unlawfully (e.f. jay walking, crossing on red, etc.). So a higher level behavior indeed.

Further, do you for any of above features also have a timeline?


I would like to propose CARLA to be the official AutoWare simulation platform. Let me explain why i think CARLA is the best option right now.

Let me start from the non-technical aspect:

  • Totally open source: code and assets are free to use for both non-commercial and commercial purposes

  • CARLA is sponsored by Intel, Toyota Research Institute, General Motors and KPIT, large corporations that help to keep CARLA aligned with the needs of the driving industry

  • In addition the CARLA challenge, is backed up by Waymo, Uber, AWS, Audi and AlphaDrive

  • CARLA has a strong community of users around the world within academia and industry

  • Multiple startups are building services around CARLA to provide advance behaviour models, simulation in the cloud, procedural generation of maps, etc. Our ecosystem is large and it keeps growing.

  • CARLA has a strong team: more than 20 engineers working full time in the platform
    We provide monthly releases with new features and bug fixes

From the perspective of technical features:

  • A modular server - multi-client architecture to distribute simulation load. Multiple clients, running in different nodes can connect to the same server. Feature releases will include a multi-server - multi-client architecture.

  • Flexible and extensible python API. CARLA itself is designed to be an API.

  • Everything can be controlled in CARLA using the API: maps, actors (vehicles, pedestrians), weather, traffic lights.

  • Full-stack simulation, from sensor simulation, planning simulation to control. Now we also expose an interface to the low-level physic parameter of vehicles, so that users can define frictions, mass, etc.

  • Standard mechanism to define maps and navigability based on OpenDrive

  • We now have a high-performance mode (sensor simulation off) that enables simulation at 800-1000 Hz. Hours of driving acquired in minutes

  • Sensor simulation: multiple cameras, LIDARs, GPS, IMU and others. Moreover, we have plans to extend these sensors with RADAR and more accurate sensors models

  • New logging and playback mode: It is possible now to log the entire simulation and resume it back at from any timestamp. This enables to try different stacks or variations of an algorithm (the simulation can be resumed and diverge from the log).

  • CARLA provides now a query engine to automatically find interesting events that happened during the simulation round. For instance, find collisions and then resume the simulation just at the interesting part.

  • Simulation can run synchronously or asynchronously. This simplifies prototyping and enables the use of a large number of sensors if the user is willing to give up real-time. Total control of the simulation step (tick).

  • We have created workflows and tools to simplify the generation of new maps. Now users can make use of RoadRunner to semi-automatically (and most recently totally automatically) create new maps. Those maps are now ready to ingest in CARLA.

  • The new 0.9.4 release can ingest maps on the fly without recompilation

  • We also have an engine for traffic simulation to generate interesting traffic scenarios based on templates and decision trees. This engine is being used for the CARLA challenge, creating pseudo-random traffic situations on the fly inspired by the NHTSA pre-crash typology.

  • CARLA supports ROS via the CARLA-ROS bridge

  • CARLA is integrated with AutoWare

  • CARLA is being integrated with Apollo

  • Headless mode. It works in docker too.

I hope this information helps.

Best regards,

Just to clarify, we won’t have a single “official” simulation platform. We intend to support as many as we can based on technical capability to use a simulator and resources to maintain a connection.

However, we will have a default simulator that will be used in tutorials and quick start guides, etc. My opinion is that licensing reasons make CARLA the best option here. @esteve has also made good arguments for why CARLA is technically the best option. As always, we are open to other opinions. In the long-run, what we want most is a healthy ecosystem of simulators that cover each other’s weaknesses.

Hi @gbiggs, we are all in the same page. That is what I meant.

Let us know how we can help.

Best regards,

We’ve been running CARLA to do comparison with LGSVL Simulator and there are few points worth mentioning:

  • CARLA performance was very poor, it was impossible to run complete Autoware or Apollo setup with CARLA in real-time
  • CARLA building and setup process was fairly complicated
  • CARLA uses Unreal engine which prohibits any commercial usage without registering all the revenues and paying 5% Unreal keeps the right to perform audits to make sure agreement terms are followed. So it does not really matter if CARLA code is under MIT license.

I’m not sure having tutorials and quick-start guides with the simulator which is unable to run Autoware in real-time, difficult to setup and requires to pay 5% if you provide it to customers is a good idea.

Dmitry Z.


@German_Ros I also agree with @gbiggs
We will not have “single” official simulator platform.
But, we want to integrate and add Documentation for some simulators in terms of usability.

@German_Ros I have some opinion to CARLA.
I try to integrate carla_bridge into Autoware and it was very very hard.

  1. Simulator is very heavy.
  2. Simulator is very very hard to build.
  3. Vehicle dynamics is strange.
  4. bridge software was not released as ROS package.
  5. Why bridge software depends PythonAPI source codes in CARLA simulator?
    Why do not you to distribute PythonAPI .egg file as binary?

In those reasons, I had to integrate carla bridge as Docker image, and I think it damages usability of carla and Autoware.
This is the feature branch for Carla Integration.


Are you referring to the ROS-bridge or to our AutoWare integration?

  1. What do you mean by CARLA being very “heavy”?

  2. The simulator is hard to build given the current instructions or just takes many steps? I mean, are there points where you got blocked and therefore need attention or is it just that since it is a UE4 project it deviates from a common ROS-based compilation recipe?

  3. Vehicle dynamics is strange sounds like a very vague description. What problems are you having with vehicle dynamics? In fact the dynamics of the vehicles are configurable by the user through the API.

  4. Right, the bridge is not a ROS package yet. We can work on making this happen

  5. The PythinAPI is a super-set of the egg. There are some additional functionalities. But we remove this dependency.

Thanks for chiming in Germán. As @gbiggs said, we are not going to have an official simulator, but a set of recommended ones. It may happen that a given simulator is better suited for a specific scenario than others and it’ll be labeled as the default, very much like what ROS2 did with the DDS vendors. For example, ROS2 has no official DDS vendor, but there’s a default one that users can just assume will work well in most cases. Users are still free to use another DDS vendor if they want (e.g. they already have a license, their usecase is very specific, they want to run ROS2 on an unsupported OS/platform, etc.)

We just want to make it easier for our users to decide which simulator to use. In order to do that, we want to ensure that all the recommended simulators are well integrated with Autoware and with good performance, they all use open standards (e.g. Lanelet2, OpenDrive), have clear licensing that doesn’t restrict its use, etc.

On the more technical side, both @Dejan_Pangercic and @gbiggs have listed many of the features we’d like simulators to have. I’d say the highlights are being able to run faster than realtime, integration with CI (including scripting and headless) and a fairly realistic physics engine, but many others are also very important.

As I said, this is not a competition, so the more simulators that are well integrated with Autoware that our users can pick from, the better.

Having a clear, well-understood license that does not restrict the use is essential for all the simulators that we intend to collaborate with. I’m not a lawyer, but it seems that the Unreal license wouldn’t be a problem here, royalties are collected by Epic only if you sell the product that uses the Unreal engine, which doesn’t seem to be the case with Carla since it’s entirely opensource. Carla has a very common license for the code (MIT) and the assets (Creative Commons-BY). That doesn’t preclude anyone (including third parties) from taking Carla, repackaging it as a proprietary product and sell it, as long as they pay royalties to Epic, but that’s not a goal the Autoware Foundation has.

If we are going down the path discussing licensing issue, let’s at least get on the same page with definitions: Open-Source != Free (free as a freedom, not as a free beer). Here is an article about that matter

@esteve Again Epic charges 5% from ALL gross revenues produced by products built with Unreal:


  • You sell Autoware to customer and include CARLA Simulation DEMO to prove it works.
    You need to pay 5% from the whole amount of the deal
  • You run Autoware and CARLA Simulator demo in the cloud and charge customers for getting access to Simulator.
    You need to pay 5% from the whole amount you are charging

Please read all the details below. Even if CARLA source code is MIT, CARLA can’t control usage of the simulator, it is very much restricted by Unreal Terms.

Generally, you are obligated to pay to Epic 5% of all gross revenue after the first $3,000 per game or application per calendar quarter, regardless of what company collects the revenue. For example, if your product earns $10 from sales on the App Store, the royalty due is $0.50 (5% of $10), even though you would receive roughly $7 from Apple after they deduct their distribution fee of roughly $3 (30% of $10).
Royalty payments are due 45 days after the close of each calendar quarter. Along with the payment, you must send a royalty report on a per-product basis. For more information, see here.

I didn’t mean to get into a debate about what Open Source means (I pasted a link to the definition of the Open Source Initiative, but was just trying to reframe the discussion to give it a more positive and constructive spin to foster collaboration from Autoware with as many simulator teams as we can.

By the way, from the link you pasted:

Despite the philosophical differences between the free-software movement and the open-source-software movement, the official definitions of free software by the FSF and of open-source software by the OSI basically refer to the same software licences, with a few minor exceptions

The Open Source Definition was copied verbatim from the Debian Free Software Guidelines, the only difference between Free Software and Open Source, is that the former is politically and philosophically motivated, whereas the latter is more about pragmatism.

As for the Unreal license, it seems fairly clear (at least to me) that only when you’re selling your game or application (i.e. you’re getting revenue from selling licenses) the royalties clause applies.

But in any case, neither you or I are lawyers, so let’s better leave it to the experts and focus on how Autoware can collaborate with the many simulation teams out there.

Not only selling the game, but ANY revenue produced with Unreal Engine triggers royalties, where the product is included. I’m posting the whole piece of the FAQ here, to remove all misunderstanding.

What do I need to do when releasing a product?

You must notify Epic when you begin collecting revenue or ship your product; see here for more details.

If I release a commercial product, what royalties are due to Epic, and when?

Generally, you are obligated to pay to Epic 5% of all gross revenue after the first $3,000 per game or application per calendar quarter, regardless of what company collects the revenue. For example, if your product earns $10 from sales on the App Store, the royalty due is $0.50 (5% of $10), even though you would receive roughly $7 from Apple after they deduct their distribution fee of roughly $3 (30% of $10).
Royalty payments are due 45 days after the close of each calendar quarter. Along with the payment, you must send a royalty report on a per-product basis. For more information, see here.

What about downloadable content, in-app purchases, microtransactions, virtual currency redemption, and SUBSCRIPTION FEES, as well as in-app advertising and affiliate program revenue?

Revenue from these sources is included in the gross revenue calculation above.

Why does Epic think it’s fair to ask for a percentage of a developer’s product revenue?

Our aim is to provide powerful tools, a scalable and productive workflow, advanced features, and millions of lines of C++ source code that enables developers to achieve more than they would otherwise be able to, so that this structure works to everyone’s benefit.

In this business model, Epic succeeds only when developers succeed using UE4. Many of the industry’s leading developers and publishers have signed up to license the Unreal Engine with royalty-based terms over the years, and now this level of access is open to everyone. And, don’t forget, we continue to offer custom terms.

Do I need to report royalties forever?

No, you only need to report royalties when you are making more than $3,000 per quarter from your product. If your game no longer is being sold, or no longer makes that amount of money, no royalty reports are due.

What if my project requires custom licensing terms?

If you require terms that reduce or eliminate the 5% royalty in exchange for an upfront fee, or if you need custom legal terms or dedicated Epic support to help your team reduce risk or achieve specific goals, we’re here to help. See the custom licensing page for details.

What if my product is released through a publisher or distributor?

You’re free to release Unreal Engine products through a publisher or distributor, and the EULA gives you the right to sublicense the necessary parts of the Unreal Engine to them so they can release your game.
When negotiating terms with publishers, please keep in mind that the royalty remains 5% of the product’s gross revenue after the first $3,000 per game per calendar quarter from users. In this scenario, feel free to refer your publisher to Epic during discussions, as it may be advantageous to all if the publisher obtains a custom-negotiated, multi-product Unreal Engine license covering your product.

What if my project wins cash awards?

You do not have to pay royalties on award winnings.

What if my product obtains crowdfunding via Kickstarter or another source?

Royalties are due on revenue from Kickstarter or other crowdfunding sources when the revenue is actually attributable to your product. For example, if the user is required to purchase a particular funding package to obtain access (now or later) to your product, or if that package gives the buyer benefits within the product such as in-game items or virtual currency.
Here’s an example of what we mean by “attributable”: Assume you provide two tiers of offers, a signed poster for $20, and a signed poster plus game access for $50. No royalties are due on ancillary products like posters, so no royalty is due on the $20 tier. On the $50 tier, the user is paying for the poster with a $20 value, and that implies that the remaining $30 of value is attributable to the product. So, for each $50 tier sale, you’d pay a royalty of $1.50 (5% of $30).

Just based on that I suggest to remove CARLA simulator from the candidates list for Autoware.Auto in order to protect final users and OURSELVES of being trapped into unexpected royalties by Epic after sharing demo build with customer.

I agree there is no competition and we should include as many simulators as possible as far as it is not posessing a danger to Autoware.Auto end users.

1 Like