2. Create Photorealistic Scenario Simulations
Simulate and visualize your vehicle models, such as an excavator, in the photorealistic 3D environment from Unreal Engine®. Use this scenario simulation to visualize vehicle movement and generate synthetic sensor data for autonomous algorithm development. By running scenario simulation in parallel with dynamic simulation in Simulink, you can validate the performance of your algorithms in perception, motion planning, and motor control.
Section 1: Three ways to perform 3D scenario simulation with Simulink and Unreal Engine
Resources
- Simulate Construction Vehicles in Unreal Engine for Material Handling – Documentation
- Using Unreal Engine with Simulink – Video Series
- Visualize with Cesium – Documentation
Convert 3D map and terrain data of a location into a scenario. - Unreal Engine Simulation Environment Requirements and Limitations – Documentation
Each version of MATLAB supports a specific version of Unreal Engine. MATLAB 2024b supports Unreal Engine Version 5.3.
Section 2: Control the machine movement in scenario simulation, and read sensor data from scenario simulation
Resources
- Use Actor Blocks to Send Control Signals to Unreal Engine – Documentation
- Use Sensor Blocks to Retrieve Sensor Data from Unreal Engine – Documentation
- Simulate Earth Moving with Autonomous Excavator in Construction Site – Documentation
In an excavator simulation, extract and process Lidar sensor data before planning excavator motion trajectory.
Section 3: Connect to other 3D Simulators using ROS
Resources
- Connectivity to ROS-Enabled Simulators (Gazebo®, Unity, CARLA, NVIDIA® Isaac Sim™) – Documentation
- Design and Simulating Autonomy for Construction Vehicles (50:56) – Video
See how to set up a scenario simulation in MATLAB and Unity Game Engine using ROS for communication.
Products
Learn about the products used to build this module of the demonstration.
