Main Content

Deep Learning Custom Training Loops

Customize deep learning training loops and loss functions

If the trainingOptions function does not provide the training options that you need for your task, or custom output layers do not support the loss functions that you need, then you can define a custom training loop. For networks that cannot be created using layer graphs, you can define custom networks as a function. To learn more, see Define Custom Training Loops, Loss Functions, and Networks.


expand all

dlnetworkDeep learning network for custom training loops
forwardCompute deep learning network output for training
predictCompute deep learning network output for inference
adamupdateUpdate parameters using adaptive moment estimation (Adam)
rmspropupdate Update parameters using root mean squared propagation (RMSProp)
sgdmupdate Update parameters using stochastic gradient descent with momentum (SGDM)
dlupdate Update parameters using custom function
minibatchqueueCreate mini-batches for deep learning
onehotencodeEncode data labels into one-hot vectors
onehotdecodeDecode probability vectors into class labels
padsequencesPad or truncate sequence data to same length
initializeInitialize learnable and state parameters of a dlnetwork
resetStateReset state parameters of neural network
dlarrayDeep learning array for custom training loops
dlgradientCompute gradients for custom training loops using automatic differentiation
dlfevalEvaluate deep learning model for custom training loops
dimsDimension labels of dlarray
finddimFind dimensions with specified label
stripdimsRemove dlarray data format
extractdataExtract data from dlarray
isdlarrayCheck if object is dlarray
functionToLayerGraphConvert deep learning model function to a layer graph
dlconvDeep learning convolution
dltranspconvDeep learning transposed convolution
lstmLong short-term memory
gruGated recurrent unit
embedEmbed discrete data
fullyconnectSum all weighted input data and apply a bias
dlode45Deep learning solution of nonstiff ordinary differential equation (ODE)
reluApply rectified linear unit activation
leakyreluApply leaky rectified linear unit activation
batchnormNormalize data across all observations for each channel independently
crosschannelnormCross channel square-normalize using local responses
groupnormNormalize data across grouped subsets of channels for each observation independently
instancenormNormalize across each channel for each observation independently
layernormNormalize data across all channels for each observation independently
avgpoolPool data to average values over spatial dimensions
maxpoolPool data to maximum value
maxunpoolUnpool the output of a maximum pooling operation
softmaxApply softmax activation to channel dimension
sigmoidApply sigmoid activation
sigmoidApply sigmoid activation
crossentropyCross-entropy loss for classification tasks
l1lossL1 loss for regression tasks
l2lossL2 loss for regression tasks
huberHuber loss for regression tasks
mseHalf mean squared error
ctcConnectionist temporal classification (CTC) loss for unaligned sequence classification
dlaccelerateAccelerate deep learning function for custom training loops
AcceleratedFunctionAccelerated deep learning function
clearCacheClear accelerated deep learning function trace cache


Custom Training Loops

Model Functions

Automatic Differentiation

Deep Learning Function Acceleration