I am having trouble using NVIDIA GeForce RTX 3090 cards for training neural networks with the Deep Learning Toolbox in MATLAB. The problems arise both as error messages and strange behaviour during the training processes of several different CNNs using two different MATLAB releases.
When using MATLAB R2020b, the following error i given when trying to start a training process for a CNN:
Error using trainNetwork (line 183)
GPU support for convolutional neural networks requires a GPU device with compute capability 3.0 or higher.
When switching to using MATLAB R2019a, the following error message occurs when e.g. training a CNN for segmentation using 256x256 image data as input and a batch size of 30 (as well as in several other cases with other types of data):
Error using trainNetwork (line 165)
Unexpected error calling cuDNN: CUDNN_STATUS_EXECUTION_FAILED.
When switching to using smaller batch sizes for this particular training process (to batch sizes 10 and 20), the training process shows a strange behaviour, with a slowly decreasing but almost static loss value (see the two images below from two completely different training processes for two different CNNs). This similar behaviour could be observed for different segmentation tasks using different data and different CNN architectures. When instead using NVIDIA TITAN RTX cards to perform these training processes, they were executed without problems, showing no similarity to each other.
Is there currently any known solution to these problems?