imagePretrainedNetwork

Pretrained neural network for images

Since R2024a

Syntax

[net,classNames] = imagePretrainedNetwork

[net,classNames] = imagePretrainedNetwork(name)

[net,classNames] = imagePretrainedNetwork(___,Name=Value)

Description

The imagePretrainedNetwork function loads a pretrained neural network and optionally adapts the neural network architecture for transfer learning and fine-tuning workflows.

[net,classNames] = imagePretrainedNetwork returns a pretrained SqueezeNet neural network and the network class names. This network is trained on the ImageNet data set for 1000 classes.

[net,classNames] = imagePretrainedNetwork(name) returns the pretrained neural network and class names for the specified pretrained neural network.

[net,classNames] = imagePretrainedNetwork(___,Name=Value) specifies additional options using one or more name-value arguments.

Examples

Load Pretrained SqueezeNet Neural Network

Load a pretrained SqueezeNet neural network and the network class names.

[net,classNames] = imagePretrainedNetwork;

View the network properties.

net

net = 
  dlnetwork with properties:

         Layers: [68x1 nnet.cnn.layer.Layer]
    Connections: [75x2 table]
     Learnables: [52x3 table]
          State: [0x3 table]
     InputNames: {'data'}
    OutputNames: {'prob_flatten'}
    Initialized: 1

  View summary with summary.

View the first few class names.

head(classNames)

    "tench"
    "goldfish"
    "great white shark"
    "tiger shark"
    "hammerhead"
    "electric ray"
    "stingray"
    "cock"

Classify Image Using Pretrained Neural Network

Load a pretrained SqueezeNet neural network.

[net,classNames] = imagePretrainedNetwork;

Read an image from a PNG file and classify it. To classify the image, first convert it to the data type single.

im = imread("peppers.png");
figure
imshow(im)

X = single(im);
scores = predict(net,X);
[label,score] = scores2label(scores,classNames);

Display the predicted label and corresponding score with the image.

figure
imshow(im)
title(string(label) + " (Score: " + score + ")")

Retrain Neural Network to Classify New Images

You can retrain a pretrained network for new datasets by adapting the neural network to match the new task and using its learned weights as a starting point. to adapt the network to the new data, replace the last few layers (known as the network head) so that it outputs prediction scores for each of the classes for the new task.

Load Training Data

Extract the MathWorks Merch data set. This is a small data set containing 75 images of MathWorks merchandise, belonging to five different classes (cap, cube, playing cards, screwdriver, and torch). The data is arranged such that the images are in subfolders that correspond to these five classes.

folderName = "MerchData";
unzip("MerchData.zip",folderName);

Create an image data store. An image datastore enables you to store large collections of image data, including data that does not fit in memory, and efficiently read batches of images during training of a neural network. Specify the folder with the extract images and indicate that the subfolder names correspond to the image labels.

imds = imageDatastore(folderName, ...
    IncludeSubfolders=true, ...
    LabelSource="foldernames");

Display some sample images.

numImages = numel(imds.Labels);
idx = randperm(numImages,16);
I = imtile(imds,Frames=idx);
figure
imshow(I)

View the class names and the number of classes.

classNames = categories(imds.Labels)

classNames = 5×1 cell
    {'MathWorks Cap'          }
    {'MathWorks Cube'         }
    {'MathWorks Playing Cards'}
    {'MathWorks Screwdriver'  }
    {'MathWorks Torch'        }

numClasses = numel(classNames)

numClasses = 5

Partition the data into training and validation data sets. Use 70% of the images for training, 15% for validation, and 15% for testing. The splitEachLabel function splits the image datastore into two new datastores.

[imdsTrain,imdsValidation,imdsTest] = splitEachLabel(imds,0.7,0.15,"randomized");

Load Pretrained Network

Load a pretrained SqueezeNet neural network. To return a neural network ready for retraining for the new data, also specify the number of classes.

net = imagePretrainedNetwork(NumClasses=numClasses)

net = 
  dlnetwork with properties:

         Layers: [68×1 nnet.cnn.layer.Layer]
    Connections: [75×2 table]
     Learnables: [52×3 table]
          State: [0×3 table]
     InputNames: {'data'}
    OutputNames: {'prob_flatten'}
    Initialized: 1

  View summary with summary.

Get the neural network input size from the input layer.

inputSize = net.Layers(1).InputSize

inputSize = 1×3

   227   227     3

The learnable layer in the network head (the last layer with learnable parameters) requires retraining. This is usually a fully connected layer or a convolutional layer with an output size that matches the number of classes.

To increase the level of updates to this layer and speed up convergence, increase the learning rate factor of its learnable parameters using the setLearnRateFactor function. Set the learning rate factors of the learnable parameters to 10.

net = setLearnRateFactor(net,"conv10/Weights",10);
net = setLearnRateFactor(net,"conv10/Bias",10);

Prepare Data for Training

The images in the datastore can have different sizes. To automatically resize the training images, use an augmented image datastore. Data augmentation also helps prevent the network from overfitting and memorizing the exact details of the training images. Specify these additional augmentation operations to perform on the training images: randomly flip the training images along the vertical axis, and randomly translate them up to 30 pixels horizontally and vertically.

pixelRange = [-30 30];

imageAugmenter = imageDataAugmenter( ...
    RandXReflection=true, ...
    RandXTranslation=pixelRange, ...
    RandYTranslation=pixelRange);

augimdsTrain = augmentedImageDatastore(inputSize(1:2),imdsTrain, ...
    DataAugmentation=imageAugmenter);

To automatically resize the validation and testing images without performing further data augmentation, use an augmented image datastore without specifying any additional preprocessing operations.

augimdsValidation = augmentedImageDatastore(inputSize(1:2),imdsValidation);
augimdsTest = augmentedImageDatastore(inputSize(1:2),imdsTest);

Specify Training Options

Specify the training options. Choosing among the options requires empirical analysis. To explore different training option configurations by running experiments, you can use the Experiment Manager app.

• Train using the Adam optimizer.

• To reduce the level of updates to the pretrained weights, use a smaller learning rate. Set the learning rate to 0.0001.

• Validate the network using the validation data every 5 iterations. For larger datasets, to prevent validation from slowing down training, increase this value.

• Display the training progress in a plot and monitor the accuracy metric.

• Disable the verbose output.

options = trainingOptions("adam", ...
    InitialLearnRate=0.0001, ...
    ValidationData=augimdsValidation, ...
    ValidationFrequency=5, ...
    Plots="training-progress", ...
    Metrics="accuracy", ...
    Verbose=false);

Train Neural Network

Train the neural network using the trainnet function. For classification, use cross-entropy loss. By default, the trainnet function uses a GPU if one is available. Using a GPU requires a Parallel Computing Toolbox™ license and a supported GPU device. For information on supported devices, see GPU Computing Requirements (Parallel Computing Toolbox). Otherwise, the trainnet function uses the CPU. To specify the execution environment, use the ExecutionEnvironment training option.

net = trainnet(augimdsTrain,net,"crossentropy",options);

Test Neural Network

Classify the validation images. To make predictions with multiple observations, use the minibatchpredict function. To covert the prediction scores to labels, use the scores2label function. The minibatchpredict function automatically uses a GPU if one is available.

YTest = minibatchpredict(net,augimdsTest);
YTest = scores2label(YTest,classNames);

Evaluate the classification accuracy for the validation data. The accuracy is the percentage of correct predictions.

TTest = imdsTest.Labels;
acc = mean(TTest==YTest)

acc = 1

Input Arguments

name — Name of pretrained neural network

"squeezenet" (default) | "googlenet" | "googlenet-places365" | "inceptionv3" | ...

Name of pretrained neural network, specified as one of these values:

imagePretrainedNetwork Model Name Argument	Neural Network Name	Depth	Size	Parameters (Millions)	Image Input Size	Required Support Package
"squeezenet"	SqueezeNet [2]	18	5.2 MB	1.24	227-by-227	None
"googlenet"	GoogLeNet [3][4]	22	27 MB	7.0	224-by-224	Deep Learning Toolbox™ Model for GoogLeNet Network
"googlenet-places365"
"inceptionv3"	Inception-v3 [5]	48	89 MB	23.9	299-by-299	Deep Learning Toolbox Model for Inception-v3 Network
"densenet201"	DenseNet-201 [6]	201	77 MB	20.0	224-by-224	Deep Learning Toolbox Model for DenseNet-201 Network
"mobilenetv2"	MobileNet-v2 [7]	53	13 MB	3.5	224-by-224	Deep Learning Toolbox Model for MobileNet-v2 Network
"resnet18"	ResNet-18 [8]	18	44 MB	11.7	224-by-224	Deep Learning Toolbox Model for ResNet-18 Network
"resnet50"	ResNet-50 [8]	50	96 MB	25.6	224-by-224	Deep Learning Toolbox Model for ResNet-50 Network
"resnet101"	ResNet-101 [8]	101	167 MB	44.6	224-by-224	Deep Learning Toolbox Model for ResNet-101 Network
"xception"	Xception [9]	71	85 MB	22.9	299-by-299	Deep Learning Toolbox Model for Xception Network
"inceptionresnetv2"	Inception-ResNet-v2 [10]	164	209 MB	55.9	299-by-299	Deep Learning Toolbox Model for Inception-ResNet-v2 Network
"shufflenet"	ShuffleNet [11]	50	5.4 MB	1.4	224-by-224	Deep Learning Toolbox Model for ShuffleNet Network
"nasnetmobile"	NASNet-Mobile [12]	*	20 MB	5.3	224-by-224	Deep Learning Toolbox Model for NASNet-Mobile Network
"nasnetlarge"	NASNet-Large [12]	*	332 MB	88.9	331-by-331	Deep Learning Toolbox Model for NASNet-Large Network
"darknet19"	DarkNet-19 [13]	19	78 MB	20.8	256-by-256	Deep Learning Toolbox Model for DarkNet-19 Network
"darknet53"	DarkNet-53 [13]	53	155 MB	41.6	256-by-256	Deep Learning Toolbox Model for DarkNet-53 Network
"efficientnetb0"	EfficientNet-b0 [14]	82	20 MB	5.3	224-by-224	Deep Learning Toolbox Model for EfficientNet-b0 Network
"alexnet"	AlexNet [15]	8	227 MB	61.0	227-by-227	Deep Learning Toolbox Model for AlexNet Network
"vgg16"	VGG-16 [16]	16	515 MB	138	224-by-224	Deep Learning Toolbox Model for VGG-16 Network
"vgg19"	VGG-19 [16]	19	535 MB	144	224-by-224	Deep Learning Toolbox Model for VGG-19 Network

Note

If you set the Weights option to "none", then for most models, downloading the support package is not required.

Name-Value Arguments

Specify optional pairs of arguments as Name1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Example: [net,classNames] = imagePretrainedNetwork("googlenet",NumClasses=10) returns a pretrained GoogLeNet neural network ready for retraining for a 10-class classification task.

NumClasses — Number of classes for classification tasks

[] (default) | positive integer

Number of classes for classification tasks, specified as a positive integer or [].

If NumClasses is an integer, then the imagePretrainedNetwork function adapts the pretrained neural network for classification tasks with the specified number of classes by replacing the learnable layer in the classification head of the network.

If you specify the NumClasses option, then NumResponses must be [] and the function must not output the classNames argument.

Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64

NumResponses — Number of responses for regression tasks

[] (default) | positive integer

Number of responses for regression tasks, specified as a positive integer or [].

If NumResponses is an integer, then the imagePretrainedNetwork function adapts the pretrained neural network for regression tasks with the specified number of responses by replacing the classification head of the network with a head for regression tasks.

If you specify the NumResponses option, then NumClasses must be [] and the function must not output the classNames argument.

Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64

Weights — Neural network weights

"pretrained" (default) | "none"

Neural network weights, specified as one of these values:

• "pretrained" — Return the neural network with its pretrained weights.

• "none" — Return the uninitialized neural network architecture only. In this case, downloading a support package is not required.

ClassNamesType — Class name type

"string" (default) | "cell"

Class name type, specified as one of these values:

• "string" — Return class names as a string array.

• "cell" — Return class names as a cell array of character vectors. Use this option for code generation workflows.

Output Arguments

net — Neural network

dlnetwork object

Neural network, returned as a dlnetwork object.

classNames — Class names

string array | cell array of character vectors

Class names, returned as a string array or a cell array of character vectors.

The function returns class names only when both the NumClasses and NumResponses options are []. The data type of classNames depends on the ClassNamesTypes option.

Data Types: string | cell

Tips

• To create and customize 2-D and 3-D ResNet neural network architectures, use the resnetNetwork and resnet3dNetwork functions, respectively.

References

[1] ImageNet. http://www.image-net.org.

[2] Iandola, Forrest N., Song Han, Matthew W. Moskewicz, Khalid Ashraf, William J. Dally, and Kurt Keutzer. "SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5 MB model size." Preprint, submitted November 4, 2016. https://arxiv.org/abs/1602.07360.

[3] Szegedy, Christian, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. "Going deeper with convolutions." In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1-9. 2015.

[4] Places. http://places2.csail.mit.edu/

[5] Szegedy, Christian, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. "Rethinking the inception architecture for computer vision." In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818-2826. 2016.

[6] Huang, Gao, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q. Weinberger. "Densely Connected Convolutional Networks." In CVPR, vol. 1, no. 2, p. 3. 2017.

[7] Sandler, M., Howard, A., Zhu, M., Zhmoginov, A. and Chen, L.C. "MobileNetV2: Inverted Residuals and Linear Bottlenecks." In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 4510-4520). IEEE.

[8] He, Kaiming, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. "Deep residual learning for image recognition." In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778. 2016.

[9] Chollet, F., 2017. "Xception: Deep Learning with Depthwise Separable Convolutions." arXiv preprint, pp.1610-02357.

[10] Szegedy, Christian, Sergey Ioffe, Vincent Vanhoucke, and Alexander A. Alemi. "Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning." In AAAI, vol. 4, p. 12. 2017.

[11] Zhang, Xiangyu, Xinyu Zhou, Mengxiao Lin, and Jian Sun. "ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices." arXiv preprint arXiv:1707.01083v2 (2017).

[12] Zoph, Barret, Vijay Vasudevan, Jonathon Shlens, and Quoc V. Le. "Learning Transferable Architectures for Scalable Image Recognition." arXiv preprint arXiv:1707.07012 2, no. 6 (2017).

[13] Redmon, Joseph. “Darknet: Open Source Neural Networks in C.” https://pjreddie.com/darknet.

[14] Mingxing Tan and Quoc V. Le, “EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks,” ArXiv Preprint ArXiv:1905.1194, 2019.

[15] Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. "ImageNet Classification with Deep Convolutional Neural Networks." Communications of the ACM 60, no. 6 (May 24, 2017): 84–90. https://doi.org/10.1145/3065386

[16] Simonyan, Karen, and Andrew Zisserman. "Very deep convolutional networks for large-scale image recognition." arXiv preprint arXiv:1409.1556 (2014).

Extended Capabilities

C/C++ Code Generation

Generate C and C++ code using MATLAB® Coder™.

Usage notes and limitations:

• Code generation only supports setting the name-value argument ClassNamesType to "cell".

• Code generation does not support setting the name-value argument Weights to "none".

GPU Code Generation

Generate CUDA® code for NVIDIA® GPUs using GPU Coder™.

Usage notes and limitations:

Refer to the usage notes and limitations in the C/C++ Code Generation section. Same limitations apply to the GPU Code Generation.

Version History

Introduced in R2024a