Generate RoadRunner Scene from Recorded Lidar Data

This example shows how to extract road information and generate a high-definition (HD) RoadRunner scene from raw lidar data.

You can create a virtual scene from recorded sensor data that represents real-world roads, and use these scenes to perform safety assessments for automated driving applications. You can also use them to localize vehicles on a map.

In this example, you:

• Detect road boundaries from recorded lidar point clouds using a pretrained deep learning model.

• Register the detected road boundaries from each point cloud to create a continuous road.

• Generate a RoadRunner HD Map from the registered road boundaries.

• Import the generated RoadRunner HD Map file into RoadRunner Scene Builder.

Load Lidar Sensor Data

This example requires the Scenario Builder for Automated Driving Toolbox™ support package. Check if the support package is installed and, if it is not installed, install it using the Get and Manage Add-Ons.

checkIfScenarioBuilderIsInstalled

Download a ZIP file containing a subset of sensor data from the PandaSet data set, and then unzip the file. This file contains data for a continuous sequence of 400 point clouds. The size of the downloaded data set is 394 MB.

dataFolder = tempdir;
dataFilename = "PandasetLidarData.zip";
url = "https://ssd.mathworks.com/supportfiles/driving/data/" + dataFilename;
filePath = fullfile(dataFolder,dataFilename); 
if ~isfile(filePath)
    websave(filePath,url)
end

ans = 
'C:\Users\rsati\AppData\Local\Temp\PandasetLidarData.zip'

unzip(filePath,dataFolder)
dataset = fullfile(dataFolder,"PandasetLidarData");

Create a datastore containing all PCD files within the specified folder by using the fileDatastore object, and read these files by using the pcread function.

lidarPath = fullfile(dataset,"Lidar");
pcds = fileDatastore(lidarPath,"ReadFcn",@pcread);

Read and display a raw point cloud from the datastore pcds. Then, return to the start of the datastore.

ptCld = read(pcds);
ax = pcshow(ptCld.Location);
zoom(ax,4)

reset(pcds)

Detect Road Boundaries

In this example, you use a pretrained deep neural network model to detect road boundary points. Download the pretrained model as a ZIP file, and then unzip the file. The size of the downloaded model is 21 MB.

modelFilename = "LidarRoadBoundaryDetectorSalsaNext.zip";
modelUrl = "https://ssd.mathworks.com/supportfiles/driving/data/" + modelFilename;
filePath = fullfile(dataFolder,modelFilename);
if ~isfile(filePath)
    websave(filePath,modelUrl)
end

ans = 
'C:\Users\rsati\AppData\Local\Temp\LidarRoadBoundaryDetectorSalsaNext.zip'

modelFolder = fullfile(dataFolder,"LidarRoadBoundaryDetectorSalsaNext");
unzip(filePath,modelFolder)
model = load(fullfile(modelFolder,"trainedRoadBoundaryModel.mat"));
roadBoundaryDetector = model.net;

Detect road boundaries by using the helperDetectRoadBoundaries function. Note that, depending on your hardware configuration, this function can take a significant amount of time to run. To reduce the execution time, limit the number of point clouds numPtClouds to 80.

numPtClouds = 80;
detectedRoadBoundaries = helperDetectRoadBoundaries(roadBoundaryDetector,pcds,numPtClouds);

detectedRoadBoundaries is a pointCloud array of size 1-by-M containing detected road boundary points. M is the number of point clouds.

Note: In this example, the pretrained deep neural network model detects only the road boundary points. It does not detect or classify the lane boundaries.

Display the road boundaries on the point cloud by using the pcshow function.

ax = pcshow(detectedRoadBoundaries(1));
zoom(ax,4)

Register Point Clouds

Register the point clouds and detected road boundary points by using the helperRegisterPointClouds function.

[registeredPtCloud,roadBoundaryLabels,egoWaypoints] = helperRegisterPointClouds(detectedRoadBoundaries);
roadBoundaries = registeredPtCloud.select(roadBoundaryLabels);

Save the registered point cloud as a PCD file to enable you to import it into RoadRunner.

pcwrite(registeredPtCloud,"RegisteredPointCloud.pcd",Encoding="compressed")

To create a RoadRunner HD Map, you must specify individual lane boundaries. Therefore, separate the detected road boundary points into leftBoundaries and rightBoundaries using the helperSegregrateRoadBoundaries function.

[leftBoundaries,rightBoundaries] = helperSegregrateRoadBoundaries(roadBoundaries,egoWaypoints);

Plot the road boundaries, and overlay the left and right lane boundaries on the road to highlight the boundary detections.

figure
pcshow(roadBoundaries)
hold on
plot3(rightBoundaries(:,1),rightBoundaries(:,2),rightBoundaries(:,3), "yo")
plot3(leftBoundaries(:,1),leftBoundaries(:,2),leftBoundaries(:,3), "go")
hold off

Create RoadRunner HD Map

Create a RoadRunner HD Map by using the roadrunnerHDMap object.

rrMap = roadrunnerHDMap;

A RoadRunner HD Map uses lane centers and lane boundaries to model roads. Calculate the lane centers by using leftBoundaries and rightBoundaries.

laneCenter = (leftBoundaries + rightBoundaries)*0.5;
rrMap.Lanes(1) = roadrunner.hdmap.Lane(ID="Lane1",Geometry=laneCenter, ...
                                       LaneType="Driving",TravelDirection="Forward");

Add leftBoundaries and rightBoundaries as lane boundaries to the map.

rrMap.LaneBoundaries(1) = roadrunner.hdmap.LaneBoundary(ID="LaneBoundary1",Geometry=leftBoundaries);
leftBoundary(rrMap.Lanes,"LaneBoundary1",Alignment="Forward")

ans = 
  AlignedReference with properties:

    Reference: [1×1 roadrunner.hdmap.Reference]
    Alignment: Forward

rrMap.LaneBoundaries(2) = roadrunner.hdmap.LaneBoundary(ID="LaneBoundary2",Geometry=rightBoundaries);
rightBoundary(rrMap.Lanes,"LaneBoundary2",Alignment="Forward")

ans = 
  AlignedReference with properties:

    Reference: [1×1 roadrunner.hdmap.Reference]
    Alignment: Forward

Visualize the HD Map roads.

plot(rrMap)

Write the RoadRunner HD Map to a binary file, which you can import into RoadRunner.

write(rrMap,"RoadRunnerHDMapRoadsFromLidar.rrhd")

You can import the RoadRunnerHDMapRoadsFromLidar.rrhd data to RoadRunner and build the scene by using the Scene Builder Tool. For detailed instructions on importing a RoadRunner HD Map file with the .rrhd extension into RoadRunner, previewing the map, and building the scene, see Build Scene (RoadRunner). You can import the registered point cloud PCD file into RoadRunner for visual validation of generated roads with respect to the imported point clouds by using the Point Cloud Tool (RoadRunner).

This figure shows a scene built using RoadRunner Scene Builder and road scene with overlaid point clouds.

Note: Use of the Scene Builder Tool requires a RoadRunner Scene Builder license.