Main Content

Custom Training Loops

Customize deep learning training loops and loss functions for sequence and tabular data

If the trainingOptions function does not provide the training options that you need for your task, or you have a loss function that the trainnet function does not support, then you can define a custom training loop. For models that cannot be specified as networks of layers, you can define the model as a function. To learn more, see Define Custom Training Loops, Loss Functions, and Networks.


expand all

dlnetworkDeep learning neural network (Since R2019b)
trainingProgressMonitorMonitor and plot training progress for deep learning custom training loops (Since R2022b)
minibatchqueueCreate mini-batches for deep learning (Since R2020b)
padsequencesPad or truncate sequence data to same length (Since R2021a)
dlarrayDeep learning array for customization (Since R2019b)
dlgradientCompute gradients for custom training loops using automatic differentiation (Since R2019b)
dlfevalEvaluate deep learning model for custom training loops (Since R2019b)
crossentropyCross-entropy loss for classification tasks (Since R2019b)
l1lossL1 loss for regression tasks (Since R2021b)
l2lossL2 loss for regression tasks (Since R2021b)
huberHuber loss for regression tasks (Since R2021a)
mseHalf mean squared error (Since R2019b)
ctcConnectionist temporal classification (CTC) loss for unaligned sequence classification (Since R2021a)
dlconvDeep learning convolution (Since R2019b)
dltranspconvDeep learning transposed convolution (Since R2019b)
lstmLong short-term memory (Since R2019b)
gruGated recurrent unit (Since R2020a)
attentionDot-product attention (Since R2022b)
embedEmbed discrete data (Since R2020b)
fullyconnectSum all weighted input data and apply a bias (Since R2019b)
dlode45Deep learning solution of nonstiff ordinary differential equation (ODE) (Since R2021b)
batchnormNormalize data across all observations for each channel independently (Since R2019b)
crosschannelnormCross channel square-normalize using local responses (Since R2020a)
groupnormNormalize data across grouped subsets of channels for each observation independently (Since R2020b)
instancenormNormalize across each channel for each observation independently (Since R2021a)
layernormNormalize data across all channels for each observation independently (Since R2021a)
avgpoolPool data to average values over spatial dimensions (Since R2019b)
maxpoolPool data to maximum value (Since R2019b)
maxunpoolUnpool the output of a maximum pooling operation (Since R2019b)
reluApply rectified linear unit activation (Since R2019b)
leakyreluApply leaky rectified linear unit activation (Since R2019b)
geluApply Gaussian error linear unit (GELU) activation (Since R2022b)
softmaxApply softmax activation to channel dimension (Since R2019b)
sigmoidApply sigmoid activation (Since R2019b)


Custom Training Loops

Automatic Differentiation