Why does gpuDevice take several minutes to complete?

I am running R2016b (also tried R2015b) on CentOS 6.8 with CUDA Toolkit 8.0 installed. I also tried driver versions 367.57 and 370.28. The machine has a NVIDIA Titan X (Pascal) GPU installed. I can use the GPU fine using gpuArray etc, but every time I call gpuDevice the call takes sevral minutes to complete. It does finally give the correct answer, though. When I call it again, it completes immediately until I restart matlab, start another instance of matlab, or call gpuDevice([]). The NVIDIA SDK examples all run fine and without delay. What could be the cause for this and how do I fix it?

Thanks!

Answer 1

[Soma Ardhanareeswaran]

The first time in the MATLAB session, 'gpuDevice' would search for available GPU devices and store the information for that session. This information is reused throughout the session. The scanning might take some amount of time depending on the system.

Another possible reason might be the license checkout time. If you have multiple license files, or multiple network license files, then the time taken to verify the license while gpuDevice is invoked the first time might be considerable.

To fix this, remove any invalid files that are presented as part of the following output:

>>feature lmsearchpath

This lists the set of local license files present. Delete the outdated ones and retain only whats required.

Also, check the environment variables 'LM_LICENSE_FILE' and 'MLM_LICENSE_FILE' and remove any old references. Please refer - https://www.mathworks.com/matlabcentral/answers/93105-what-are-the-lm_license_file-and-mlm_license_file-environment-variables-and-how-do-i-set-or-check-fo