Semantic Segmentation of Multispectral Images by Using Quantized U-Net on FPGA
This example show how to use the Deep Learning HDL Toolbox™ to deploy a quantized U-Net to perform semantic segmentation on multispectral images. The example uses the pretrained U-Net network to demonstrate quantization and deployment of the quantized network. Quantization helps reduce the memory requirement of a deep neural network by quantizing weights, biases, and activations of network layers to 8-bit scaled integer data types. To retrieve the prediction results, use MATLAB®.
Deploy the quantized U-Net network by creating a dlhdl.Workflow object. Use the dlhdl.Workflow object to:
Generate a list of instructions, weights and biases by using the
compilemethod.Generate a programming file for the FPGA by using the
deploymethod.Retrieve the network prediction results and performance by using the
predictmethod.
The quantized network takes in a multispectral input image of size 256-by-256 that has six channels and outputs a segmentation map where each pixel corresponds to one of 18 classes. This network is taken from the Semantic Segmentation of Multispectral Images Using Deep Learning example from the Computer Vision Toolbox™. To train the network, see Semantic Segmentation of Multispectral Images Using Deep Learning (Image Processing Toolbox).
Prerequisites
Deep Learning Toolbox™
Deep Learning HDL Toolbox™
Deep Learning HDL Toolbox™ Support Package for Intel FPGA and SoC
Intel Arria10 SoC Development Kit
Deep Learning Toolbox™ Model Quantization Library Support Package.
MATLAB Coder Interface for Deep learning
Load Pretrained U-Net Network
Load the pretrained Directed Acyclic Graph (DAG) network U-Net using the downloadTrainedUnet helper function. This function is attached to the example as a supporting file.
imageDir = tempdir; trainedUNetURL = 'https://www.mathworks.com/supportfiles/vision/data/multispectralUnet.mat'; downloadTrainedUnet(trainedUNetURL, imageDir); load(fullfile(imageDir, 'trainedUnet', 'multispectralUnet.mat'));
To obtain information about the 58 layers in the DAG network, use the analyzeNetwork function.
analyzeNetwork(net)
Download Data
The pretrained network was trained on a high-resolution multispectral data set [1]. The image set was captured using a drone over Hamlin Beach State Park, NY. The data contains labeled training, validation, and test set that have 18 object class labels. The size of the data file is ~3.0 GB. For calibration and testing of the network, use parts of the training data set.
Download the MAT-file version of the data set by using the downloadHamlinBeachMSIData helper function. This function is attached to the example as a supporting file.
imageDir = tempdir;
url = 'https://home.cis.rit.edu/~cnspci/other/data/rit18_data.mat';
downloadHamlinBeachMSIData(url, imageDir);Create Calibration Data
The pretrained U-Net network accepts inputs of size 256-by-256-by-6. The training data in the downloaded MAT file has a size of 7-by-9393-by-5642. Use the extractMultispectralData helper function to extract patches of size 256-by-256-by-6 and store them in MAT files for calibration. The seventh channel in the training data is a binary mask and is not used by the pretrained network for inference.
For best quantization results, the calibration data must be representative of actual inputs that are predicted by the U-Net network. Expedite the calibration process by reducing the calibration data set to six images. Choose the six images so that they form a 2-by-3 grid to represent a large continuous image.
foldername = 'CalibData'; dataPath = fullfile(imageDir, 'rit18_data', 'rit18_data.mat'); im = extractMultispectralData(foldername, dataPath, 2, 3);
The first three channels of the multispectral training data contain RGB information. Display a histogram-equalized version of the extracted data.
im = histeq(im(:,:,[3 2 1]));
montage({im});
Create an imageDatastore object to use for calibration. The patches are loaded from the folder 'CalibData'.
imds = imageDatastore('CalibData', FileExtensions = '.mat', ReadFcn = @matReader);
Create dlquantizer Object
Create a quantized network object by using dlquantizer. Set the target execution environment to FPGA.
dlQuantObj = dlquantizer(net, ExecutionEnvironment = 'FPGA');Calibrate Quantized Network
Use the calibrate function to exercise the network by using sample inputs and collect the range information. The calibrate function exercises the network. The function collects the dynamic ranges of the weights and biases in the convolution and fully connected layers of the network and in the dynamic ranges of the activations in all layers of the network. The calibrate function returns a table.
calibrate(dlQuantObj, imds)
ans=103×5 table
Optimized Layer Name Network Layer Name Learnables / Activations MinValue MaxValue
____________________________________ ____________________________ ________________________ _________ _________
{'Encoder-Section-1-Conv-1_Weights'} {'Encoder-Section-1-Conv-1'} "Weights" -0.078472 0.083924
{'Encoder-Section-1-Conv-1_Bias' } {'Encoder-Section-1-Conv-1'} "Bias" 0.7125 1.1249
{'Encoder-Section-1-Conv-2_Weights'} {'Encoder-Section-1-Conv-2'} "Weights" -0.23892 0.24892
{'Encoder-Section-1-Conv-2_Bias' } {'Encoder-Section-1-Conv-2'} "Bias" 0.70602 1.381
{'Encoder-Section-2-Conv-1_Weights'} {'Encoder-Section-2-Conv-1'} "Weights" -0.048319 0.075386
{'Encoder-Section-2-Conv-1_Bias' } {'Encoder-Section-2-Conv-1'} "Bias" 0.93696 1.049
{'Encoder-Section-2-Conv-2_Weights'} {'Encoder-Section-2-Conv-2'} "Weights" -0.18248 0.19105
{'Encoder-Section-2-Conv-2_Bias' } {'Encoder-Section-2-Conv-2'} "Bias" 0.85737 1.0482
{'Encoder-Section-3-Conv-1_Weights'} {'Encoder-Section-3-Conv-1'} "Weights" -0.012313 0.027855
{'Encoder-Section-3-Conv-1_Bias' } {'Encoder-Section-3-Conv-1'} "Bias" 0.9723 1.0495
{'Encoder-Section-3-Conv-2_Weights'} {'Encoder-Section-3-Conv-2'} "Weights" -0.14617 0.13171
{'Encoder-Section-3-Conv-2_Bias' } {'Encoder-Section-3-Conv-2'} "Bias" 0.96037 1.0234
{'Encoder-Section-4-Conv-1_Weights'} {'Encoder-Section-4-Conv-1'} "Weights" -0.006589 0.0069637
{'Encoder-Section-4-Conv-1_Bias' } {'Encoder-Section-4-Conv-1'} "Bias" 0.9854 1.0057
{'Encoder-Section-4-Conv-2_Weights'} {'Encoder-Section-4-Conv-2'} "Weights" -0.10257 0.1018
{'Encoder-Section-4-Conv-2_Bias' } {'Encoder-Section-4-Conv-2'} "Bias" 0.99395 1.0025
⋮
Create Target Object
Set the synthesis tool path to point to an installed Intel® Quartus® Prime Standard Edition 20.1 executable file. You must have already installed Altera® Quartus II.
% hdlsetuptoolpath(ToolName = 'Altera Quartus II', ToolPath = 'C:\intel\20.1\quartus\bin\quartus.exe');Create a target object that has a custom name for your target device and an interface to connect your target device to the host computer. Interface options are JTAG (default) and Ethernet.
hTarget = dlhdl.Target('Intel', Interface = 'JTAG');
Create Workflow Object
Create an object of the dlhdl.Workflow class. Specify the network and bitstream name. Specify the quantized network object dlQuantObj as the network. Make sure that the bitstream name matches the data type and the FPGA board that you are targeting. In this example, the target FPGA board is the Intel Arria 10 SoC board. The bitstream uses an int8 data type.
hW = dlhdl.Workflow(Network = dlQuantObj, Bitstream = 'arria10soc_int8', Target = hTarget);Compile Workflow Object
To compile the U-Net network, run the compile function of the dlhdl.Workflow object.
dn = compile(hW)
### Compiling network for Deep Learning FPGA prototyping ...
### Targeting FPGA bitstream arria10soc_int8.
### The network includes the following layers:
1 'ImageInputLayer' Image Input 256×256×6 images with 'zerocenter' normalization (SW Layer)
2 'Encoder-Section-1-Conv-1' Convolution 64 3×3×6 convolutions with stride [1 1] and padding [1 1 1 1] (HW Layer)
3 'Encoder-Section-1-ReLU-1' ReLU ReLU (HW Layer)
4 'Encoder-Section-1-Conv-2' Convolution 64 3×3×64 convolutions with stride [1 1] and padding [1 1 1 1] (HW Layer)
5 'Encoder-Section-1-ReLU-2' ReLU ReLU (HW Layer)
6 'Encoder-Section-1-MaxPool' Max Pooling 2×2 max pooling with stride [2 2] and padding [0 0 0 0] (HW Layer)
7 'Encoder-Section-2-Conv-1' Convolution 128 3×3×64 convolutions with stride [1 1] and padding [1 1 1 1] (HW Layer)
8 'Encoder-Section-2-ReLU-1' ReLU ReLU (HW Layer)
9 'Encoder-Section-2-Conv-2' Convolution 128 3×3×128 convolutions with stride [1 1] and padding [1 1 1 1] (HW Layer)
10 'Encoder-Section-2-ReLU-2' ReLU ReLU (HW Layer)
11 'Encoder-Section-2-MaxPool' Max Pooling 2×2 max pooling with stride [2 2] and padding [0 0 0 0] (HW Layer)
12 'Encoder-Section-3-Conv-1' Convolution 256 3×3×128 convolutions with stride [1 1] and padding [1 1 1 1] (HW Layer)
13 'Encoder-Section-3-ReLU-1' ReLU ReLU (HW Layer)
14 'Encoder-Section-3-Conv-2' Convolution 256 3×3×256 convolutions with stride [1 1] and padding [1 1 1 1] (HW Layer)
15 'Encoder-Section-3-ReLU-2' ReLU ReLU (HW Layer)
16 'Encoder-Section-3-MaxPool' Max Pooling 2×2 max pooling with stride [2 2] and padding [0 0 0 0] (HW Layer)
17 'Encoder-Section-4-Conv-1' Convolution 512 3×3×256 convolutions with stride [1 1] and padding [1 1 1 1] (HW Layer)
18 'Encoder-Section-4-ReLU-1' ReLU ReLU (HW Layer)
19 'Encoder-Section-4-Conv-2' Convolution 512 3×3×512 convolutions with stride [1 1] and padding [1 1 1 1] (HW Layer)
20 'Encoder-Section-4-ReLU-2' ReLU ReLU (HW Layer)
21 'Encoder-Section-4-DropOut' Dropout 50% dropout (HW Layer)
22 'Encoder-Section-4-MaxPool' Max Pooling 2×2 max pooling with stride [2 2] and padding [0 0 0 0] (HW Layer)
23 'Mid-Conv-1' Convolution 1024 3×3×512 convolutions with stride [1 1] and padding [1 1 1 1] (HW Layer)
24 'Mid-ReLU-1' ReLU ReLU (HW Layer)
25 'Mid-Conv-2' Convolution 1024 3×3×1024 convolutions with stride [1 1] and padding [1 1 1 1] (HW Layer)
26 'Mid-ReLU-2' ReLU ReLU (HW Layer)
27 'Mid-DropOut' Dropout 50% dropout (HW Layer)
28 'Decoder-Section-1-UpConv' Transposed Convolution 512 2×2×1024 transposed convolutions with stride [2 2] and cropping [0 0 0 0] (HW Layer)
29 'Decoder-Section-1-UpReLU' ReLU ReLU (HW Layer)
30 'Decoder-Section-1-DepthConcatenation' Depth concatenation Depth concatenation of 2 inputs (HW Layer)
31 'Decoder-Section-1-Conv-1' Convolution 512 3×3×1024 convolutions with stride [1 1] and padding [1 1 1 1] (HW Layer)
32 'Decoder-Section-1-ReLU-1' ReLU ReLU (HW Layer)
33 'Decoder-Section-1-Conv-2' Convolution 512 3×3×512 convolutions with stride [1 1] and padding [1 1 1 1] (HW Layer)
34 'Decoder-Section-1-ReLU-2' ReLU ReLU (HW Layer)
35 'Decoder-Section-2-UpConv' Transposed Convolution 256 2×2×512 transposed convolutions with stride [2 2] and cropping [0 0 0 0] (HW Layer)
36 'Decoder-Section-2-UpReLU' ReLU ReLU (HW Layer)
37 'Decoder-Section-2-DepthConcatenation' Depth concatenation Depth concatenation of 2 inputs (HW Layer)
38 'Decoder-Section-2-Conv-1' Convolution 256 3×3×512 convolutions with stride [1 1] and padding [1 1 1 1] (HW Layer)
39 'Decoder-Section-2-ReLU-1' ReLU ReLU (HW Layer)
40 'Decoder-Section-2-Conv-2' Convolution 256 3×3×256 convolutions with stride [1 1] and padding [1 1 1 1] (HW Layer)
41 'Decoder-Section-2-ReLU-2' ReLU ReLU (HW Layer)
42 'Decoder-Section-3-UpConv' Transposed Convolution 128 2×2×256 transposed convolutions with stride [2 2] and cropping [0 0 0 0] (HW Layer)
43 'Decoder-Section-3-UpReLU' ReLU ReLU (HW Layer)
44 'Decoder-Section-3-DepthConcatenation' Depth concatenation Depth concatenation of 2 inputs (HW Layer)
45 'Decoder-Section-3-Conv-1' Convolution 128 3×3×256 convolutions with stride [1 1] and padding [1 1 1 1] (HW Layer)
46 'Decoder-Section-3-ReLU-1' ReLU ReLU (HW Layer)
47 'Decoder-Section-3-Conv-2' Convolution 128 3×3×128 convolutions with stride [1 1] and padding [1 1 1 1] (HW Layer)
48 'Decoder-Section-3-ReLU-2' ReLU ReLU (HW Layer)
49 'Decoder-Section-4-UpConv' Transposed Convolution 64 2×2×128 transposed convolutions with stride [2 2] and cropping [0 0 0 0] (HW Layer)
50 'Decoder-Section-4-UpReLU' ReLU ReLU (HW Layer)
51 'Decoder-Section-4-DepthConcatenation' Depth concatenation Depth concatenation of 2 inputs (HW Layer)
52 'Decoder-Section-4-Conv-1' Convolution 64 3×3×128 convolutions with stride [1 1] and padding [1 1 1 1] (HW Layer)
53 'Decoder-Section-4-ReLU-1' ReLU ReLU (HW Layer)
54 'Decoder-Section-4-Conv-2' Convolution 64 3×3×64 convolutions with stride [1 1] and padding [1 1 1 1] (HW Layer)
55 'Decoder-Section-4-ReLU-2' ReLU ReLU (HW Layer)
56 'Final-ConvolutionLayer' Convolution 18 1×1×64 convolutions with stride [1 1] and padding [0 0 0 0] (HW Layer)
57 'Softmax-Layer' Softmax softmax (HW Layer)
58 'Segmentation-Layer' Pixel Classification Layer Cross-entropy loss with 'Road Markings', 'Tree', and 16 other classes (SW Layer)
### Notice: The layer 'Decoder-Section-1-UpConv' of type 'nnet.cnn.layer.TransposedConvolution2DLayer' is split into an image input layer 'Decoder-Section-1-UpConv_insertZeros' and an addition layer 'Decoder-Section-1-UpConv' for normalization on hardware.
### Notice: The layer 'Decoder-Section-2-UpConv' of type 'nnet.cnn.layer.TransposedConvolution2DLayer' is split into an image input layer 'Decoder-Section-2-UpConv_insertZeros' and an addition layer 'Decoder-Section-2-UpConv' for normalization on hardware.
### Notice: The layer 'Decoder-Section-3-UpConv' of type 'nnet.cnn.layer.TransposedConvolution2DLayer' is split into an image input layer 'Decoder-Section-3-UpConv_insertZeros' and an addition layer 'Decoder-Section-3-UpConv' for normalization on hardware.
### Notice: The layer 'Decoder-Section-4-UpConv' of type 'nnet.cnn.layer.TransposedConvolution2DLayer' is split into an image input layer 'Decoder-Section-4-UpConv_insertZeros' and an addition layer 'Decoder-Section-4-UpConv' for normalization on hardware.
### Notice: The layer 'ImageInputLayer' with type 'nnet.cnn.layer.ImageInputLayer' is implemented in software.
### Notice: The layer 'Softmax-Layer' with type 'nnet.cnn.layer.SoftmaxLayer' is implemented in software.
### Notice: The layer 'Segmentation-Layer' with type 'nnet.cnn.layer.PixelClassificationLayer' is implemented in software.
### Compiling layer group: Encoder-Section-1-Conv-1>>Encoder-Section-1-ReLU-2 ...
### Compiling layer group: Encoder-Section-1-Conv-1>>Encoder-Section-1-ReLU-2 ... complete.
### Compiling layer group: Encoder-Section-1-MaxPool>>Encoder-Section-2-ReLU-2 ...
### Compiling layer group: Encoder-Section-1-MaxPool>>Encoder-Section-2-ReLU-2 ... complete.
### Compiling layer group: Encoder-Section-2-MaxPool>>Encoder-Section-3-ReLU-2 ...
### Compiling layer group: Encoder-Section-2-MaxPool>>Encoder-Section-3-ReLU-2 ... complete.
### Compiling layer group: Encoder-Section-3-MaxPool>>Encoder-Section-4-ReLU-2 ...
### Compiling layer group: Encoder-Section-3-MaxPool>>Encoder-Section-4-ReLU-2 ... complete.
### Compiling layer group: Encoder-Section-4-MaxPool>>Mid-ReLU-2 ...
### Compiling layer group: Encoder-Section-4-MaxPool>>Mid-ReLU-2 ... complete.
### Compiling layer group: Decoder-Section-1-UpConv_insertZeros ...
### Compiling layer group: Decoder-Section-1-UpConv_insertZeros ... complete.
### Compiling layer group: Decoder-Section-1-UpConv>>Decoder-Section-1-UpReLU ...
### Compiling layer group: Decoder-Section-1-UpConv>>Decoder-Section-1-UpReLU ... complete.
### Compiling layer group: Decoder-Section-1-Conv-1>>Decoder-Section-1-ReLU-2 ...
### Compiling layer group: Decoder-Section-1-Conv-1>>Decoder-Section-1-ReLU-2 ... complete.
### Compiling layer group: Decoder-Section-2-UpConv_insertZeros ...
### Compiling layer group: Decoder-Section-2-UpConv_insertZeros ... complete.
### Compiling layer group: Decoder-Section-2-UpConv>>Decoder-Section-2-UpReLU ...
### Compiling layer group: Decoder-Section-2-UpConv>>Decoder-Section-2-UpReLU ... complete.
### Compiling layer group: Decoder-Section-2-Conv-1>>Decoder-Section-2-ReLU-2 ...
### Compiling layer group: Decoder-Section-2-Conv-1>>Decoder-Section-2-ReLU-2 ... complete.
### Compiling layer group: Decoder-Section-3-UpConv_insertZeros ...
### Compiling layer group: Decoder-Section-3-UpConv_insertZeros ... complete.
### Compiling layer group: Decoder-Section-3-UpConv>>Decoder-Section-3-UpReLU ...
### Compiling layer group: Decoder-Section-3-UpConv>>Decoder-Section-3-UpReLU ... complete.
### Compiling layer group: Decoder-Section-3-Conv-1>>Decoder-Section-3-ReLU-2 ...
### Compiling layer group: Decoder-Section-3-Conv-1>>Decoder-Section-3-ReLU-2 ... complete.
### Compiling layer group: Decoder-Section-4-UpConv_insertZeros ...
### Compiling layer group: Decoder-Section-4-UpConv_insertZeros ... complete.
### Compiling layer group: Decoder-Section-4-UpConv>>Decoder-Section-4-UpReLU ...
### Compiling layer group: Decoder-Section-4-UpConv>>Decoder-Section-4-UpReLU ... complete.
### Compiling layer group: Decoder-Section-4-Conv-1>>Final-ConvolutionLayer ...
### Compiling layer group: Decoder-Section-4-Conv-1>>Final-ConvolutionLayer ... complete.
### Allocating external memory buffers:
offset_name offset_address allocated_space
_______________________ ______________ _________________
"InputDataOffset" "0x00000000" "16.0 MB"
"OutputResultOffset" "0x01000000" "48.0 MB"
"SchedulerDataOffset" "0x04000000" "24.0 MB"
"SystemBufferOffset" "0x05800000" "28.0 MB"
"InstructionDataOffset" "0x07400000" "36.0 MB"
"ConvWeightDataOffset" "0x09800000" "540.0 MB"
"EndOffset" "0x2b400000" "Total: 692.0 MB"
### Network compilation complete.
dn = struct with fields:
weights: [1×1 struct]
instructions: [1×1 struct]
registers: [1×1 struct]
syncInstructions: [1×1 struct]
constantData: {}
Program Bitstream into FPGA and Download Network Weights
To deploy the network on the Intel Arria10 SoC hardware, run the deploy function of the dlhdl.Workflow object. This function uses the output of the compile function to program the FPGA board by using the programming file. The function also loads the network weights and biases into the device. The deploy function starts programming the FPGA device, displays progress messages, and the time it takes to deploy the network.
deploy(hW)
### Programming FPGA Bitstream using JTAG... ### Programming the FPGA bitstream has been completed successfully. ### Loading weights to Conv Processor. ### Conv Weights loaded. Current time is 14-Dec-2021 23:40:29
Load Example Images
Extract patches for inference on FPGA by using the extractMultispectralData helper function and store them in MAT files. Create 20 patches of size 256-by-256-by-6 so that they form a 4-by-5 grid to represent a large input image.
foldername = 'TestData'; dataPath = fullfile(imageDir, 'rit18_data', 'rit18_data.mat'); extractMultispectralData(foldername, dataPath, 4, 5);
Load the extracted data into testData by using the helperConcatenateMultispectralData helper function. It concatenates inputs along the fourth dimension for multiframe prediction by using the dlhdl.Workflow object. The function is attached to the example as a supporting file.
testData = helperConcatenateMultispectralData(foldername);
Run Prediction
Execute the predict function of the dlhdl.Workflow object and display the prediction results for testData. Because the input is concatenated along the fourth dimension, the predictions occur simultaneously.
[prediction, speed] = predict(hW, testData(:,:,1:6,:), 'Profile', 'on');
### Finished writing input activations.
### Running in multi-frame mode with 20 inputs.
Deep Learning Processor Profiler Performance Results
LastFrameLatency(cycles) LastFrameLatency(seconds) FramesNum Total Latency Frames/s
------------- ------------- --------- --------- ---------
Network 175391449 1.16928 20 3507877237 0.9
Encoder-Section-1-Conv-1 1216888 0.00811
Encoder-Section-1-Conv-2 2898182 0.01932
Encoder-Section-1-MaxPool 5225243 0.03483
Encoder-Section-2-Conv-1 689902 0.00460
Encoder-Section-2-Conv-2 2604963 0.01737
Encoder-Section-2-MaxPool 4862763 0.03242
Encoder-Section-3-Conv-1 416523 0.00278
Encoder-Section-3-Conv-2 2406534 0.01604
Encoder-Section-3-MaxPool 6432961 0.04289
Encoder-Section-4-Conv-1 345878 0.00231
Encoder-Section-4-Conv-2 4062950 0.02709
Encoder-Section-4-MaxPool 7270617 0.04847
Mid-Conv-1 1298161 0.00865
Mid-Conv-2 14902377 0.09935
Decoder-Section-1-UpConv_insertZeros 14894578 0.09930
Decoder-Section-1-UpConv 6431694 0.04288
Decoder-Section-1-Conv-1 1842230 0.01228
Decoder-Section-1-Conv-2 9572771 0.06382
Decoder-Section-2-UpConv_insertZeros 10785828 0.07191
Decoder-Section-2-UpConv 4863034 0.03242
Decoder-Section-2-Conv-1 3103690 0.02069
Decoder-Section-2-Conv-2 10455339 0.06970
Decoder-Section-3-UpConv_insertZeros 10361041 0.06907
Decoder-Section-3-UpConv 5225305 0.03484
Decoder-Section-3-Conv-1 4555619 0.03037
Decoder-Section-3-Conv-2 11171105 0.07447
Decoder-Section-4-UpConv_insertZeros 11466232 0.07644
Decoder-Section-4-UpConv 5907915 0.03939
Decoder-Section-4-Conv-1 2673353 0.01782
Decoder-Section-4-Conv-2 1539401 0.01026
Final-ConvolutionLayer 5908123 0.03939
* The clock frequency of the DL processor is: 150MHz
The output of hW.predict is of shape 256-by-256-by-18-by-20, where the outputs are concatenated along the fourth dimension. The 20 test images were created from a 1024-by-1280-by-6 section of the training data. The inputs and outputs are rearranged by using helperArrangeInput and helperArrangeOutput functions to display the prediction results. The functions are attached to the example as supporting files.
testImage = helperArrangeInput(testData, 4, 5); segmentedImage = helperArrangeOutput(prediction, 4, 5);
Display the Prediction Results
Overlay the segmented image on the histogram-equalized RGB test image and display the prediction results.
classNames = [ ... "RoadMarkings", "Tree", "Building", "Vehicle", "Person", ... "LifeguardChair", "PicnicTable", "BlackWoodPanel", ... "WhiteWoodPanel", "OrangeLandingPad", "Buoy", "Rocks", ... "LowLevelVegetation", "Grass_Lawn", "Sand_Beach", ... "Water_Lake", "Water_Pond", "Asphalt"]; cmap = jet(numel(classNames)); N = numel(classNames); ticks = 1/(N*2):1/N:1; B = labeloverlay(histeq(testImage(:,:,[3 2 1])), medfilt2(segmentedImage), Transparency = 0.4, Colormap = cmap); figure imshow(B); title('Labeled Test Image') colorbar('TickLabels', cellstr(classNames), 'Ticks', ticks, 'TickLength', 0, 'TickLabelInterpreter', 'none'); colormap(cmap)
References
[1] Kemker, R., C. Salvaggio, and C. Kanan. "High-Resolution Multispectral Dataset for Semantic Segmentation." CoRR, abs/1703.01918. 2017.
[2] Kemker, Ronald, Carl Salvaggio, and Christopher Kanan. "Algorithms for Semantic Segmentation of Multispectral Remote Sensing Imagery Using Deep Learning." ISPRS Journal of Photogrammetry and Remote Sensing, Deep Learning RS Data, 145 (November 1, 2018): 60-77. https://doi.org/10.1016/j.isprsjprs.2018.04.014.
See Also
dlhdl.Target | dlhdl.Workflow | dlquantizer | calibrate | compile | deploy | predict | classify
You can also select a web site from the following list
Americas
- América Latina (Español)
- Canada (English)
- United States (English)
Europe
- Belgium (English)
- Denmark (English)
- Deutschland (Deutsch)
- España (Español)
- Finland (English)
- France (Français)
- Ireland (English)
- Italia (Italiano)
- Luxembourg (English)
- Netherlands (English)
- Norway (English)
- Österreich (Deutsch)
- Portugal (English)
- Sweden (English)
- Switzerland
- United Kingdom (English)
Asia Pacific
- Australia (English)
- India (English)
- New Zealand (English)
- 中国
- 日本Japanese (日本語)
- 한국Korean (한국어)