NVIDIA Jetson setup issue: coder.checkGpuInstall can't find nvcc

I am trying to use the GPU coder to send converted code to my Jetson Orin Nano. However, when I use the coder.checkGpuInstall command, it says that it can't find nvcc.

I followed the setup instructions with the variables here:

Documentation

and referenced the following questions before posting this one:

https://www.mathworks.com/matlabcentral/answers/506483-ncc-problem-in-jetson-nano

https://www.mathworks.com/matlabcentral/answers/2068206-checking-for-cuda-availability-on-the-target-checking-for-nvcc-in-the-target-system-path-war

This is a cropped version of my script:

if (boardName == "jetson")
    if isempty(deviceAddress)
        hwobj = jetson();
    else
        hwobj = jetson(deviceAddress,userName,password);
    end
else
    if isempty(deviceAddress)
        hwobj = drive();
    else
        hwobj = drive(deviceAddress,userName,password);
    end
end

if (boardName == "jetson")
    envCfg = coder.gpuEnvConfig('jetson'); 
else
    envCfg = coder.gpuEnvConfig('drive'); 
end
envCfg.BasicCodegen = 1; 
envCfg.HardwareObject = hwobj; 
coder.checkGpuInstall(envCfg);

and the output:

Checking for CUDA availability on the Target...
Checking for 'nvcc' in the target system path...
Checking for cuDNN library availability on the Target...
Checking for TensorRT library availability on the Target...
Checking for prerequisite libraries is complete.
Gathering hardware details...
Checking for third-party library availability on the Target...
Warning: Unable to find the SDL 1.2 library on the target. 
> In nvidiaio.internal.checkForSdlLibs (line 14)
In nvidiaboard/checkAndGetHardwareConfig
In jetson
In gpu_jetson_setup (line 10) 
Warning: Unable to find one of the packages "sox", "libsox-fmt-all" or "libsox-dev". Make sure to have installed these
packages on the target hardware using "apt-get". These are required for successful deployment of Audio File Read block
in Simulink. 
> In nvidiaio.internal.checkSoXVersion (line 17)
In nvidiaboard/checkAndGetHardwareConfig
In jetson
In gpu_jetson_setup (line 10) 
Warning: Unable to fetch information about GPU devices. 
> In nvidiaio.internal.getGpuInfo (line 77)
In nvidiaboard/getGpuInfo
In nvidiaboard/checkAndGetHardwareConfig
In jetson
In gpu_jetson_setup (line 10) 
Gathering hardware details is complete.
 Board name              : NVIDIA Jetson Orin Nano Developer Kit
 CUDA Version            : 12.2
 cuDNN Version           : 8.9
 TensorRT Version        : 8.6
 GStreamer Version       : 1.20.3
 V4L2 Version            : 1.22.1-2build1
 SDL Version             :  
 OpenCV Version          : 4.8.0
 Available Webcams       :  
 Available GPUs          :  
 Available Digital Pins  : 7  11  12  13  15  16  18  19  21  22  23  24  26  29  31  32  33  35  36  37  38  40
Compatible GPU           : FAILED (Unable to find GPU information. This is due to the missing of 'nvcc' on the system path. Update the '.bashrc' script on the target to set up the required environment variables.)
CUDA Environment         : PASSED 
	Runtime   : PASSED 
	cuFFT     : PASSED 
	cuSOLVER  : PASSED 
	cuBLAS    : PASSED 
Basic Code Generation    : PASSED 

In accordance with the setup guide, I updated both the .bashrc file (and /etc/environment file for good measure) on the jetson, I have pasted them below:

.bashrc excerpt:

# ~/.bashrc: executed by bash(1) for non-login shells.
# see /usr/share/doc/bash/examples/startup-files (in the package bash-doc)
# for examples

# If not running interactively, don't do anything
case $- in
    *i*) ;;
      *)
      export PATH=$PATH:/usr/local/cuda/bin
      export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64
      export ARM_COMPUTELIB=$ARM_COMPUTELIB:/usr/local/arm_compute
      return;;
esac

# don't put duplicate lines or lines starting with space in the history.
# See bash(1) for more options
HISTCONTROL=ignoreboth

/etc/environment:

PATH="/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin"
LANG="en_US.UTF-8"
LD_LIBRARY_PATH="/usr/local/cuda/lib64"

And to confirm, the jetson does show nvcc is installed correctly:

$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Tue_Aug_15_22:08:11_PDT_2023
Cuda compilation tools, release 12.2, V12.2.140
Build cuda_12.2.r12.2/compiler.33191640_0

Matlab version info:

MATLAB Version: 24.1.0.2628055 (R2024a) Update 4
Operating System: Linux 5.15.0-116-generic #126~20.04.1-Ubuntu SMP Mon Jul 1 15:40:07 UTC 2024 x86_64
Java Version: Java 1.8.0_202-b08 with Oracle Corporation Java HotSpot(TM) 64-Bit Server VM mixed mode

Any help on this topic would be much appreciated. Thank you!

Answer 1

[Chao Luo]

The error message

Compatible GPU : FAILED (Unable to find GPU information. This is due to the missing of 'nvcc' on the system path. Update the '.bashrc' script on the target to set up the required environment variables.)

is misleading.

Because CUDA environment check passed, "CUDA Environment : PASSED ", nvcc is setup properly.

I think the issue is about getting the GPU info. It is caused by the newer CUDA version (12.2) which removed the support of compute capability 3.5.

Basically, you can ignore the error message and remeber to set the compute capability properly in your codegen config.

[Ryan]

Thank you for the answer, and my apologies for the delay in response. I have confirmed that my GPU compute capability is '8.7'. Which attribute do I need to set in the envCfg to reflect that? Or do I set that somwhere else?

(on the jetson)

$ nvidia-smi --query-gpu=compute_cap --format=csv
compute_cap
8.7

[Chao Luo]

You cannot set the compute capability in envCfg, because it is designed to detect the GPU instead of let you tell what GPU you have. Because you are using a MATLAB release before CUDA 12, not aware of the deprecation of compute capability 3.5, it hardcoded the compute capability to 3.5. The issue will be fixed in newer MATLAB release.

For now, you can simply ignore the error from coder.checkGpuInstall.

When you do real codegen, you can set the compute capability to 8.7 like this:

cfg = coder.gpuConfig();
cfg.GpuConfig.ComputeCapability = '8.7';

Answer 2

[Abhishek Kumar Singh]

Hi @Ryan,

It seems that you have diligently followed all the setup and configuration instructions provided in the documentation and also tried the workarounds suggested in other MATLAB answers.

It is clear that nvcc is installed correctly on your Jetson board. However, one thing you might want to check is a potential mismatch in the CUDA version paths. When you run:

which nvcc

it will display the installed path of nvcc. Sometimes, the environment variables are set to use /usr/local/cuda, but in reality, the nvcc binary is located in a version-specific folder like /usr/local/cuda-12.2/bin.

To resolve this issue, you need to update your environment variables to point to the correct CUDA version directory.

You can update the CUDA paths as:

# ~/.bashrc: executed by bash(1) for non-login shells.
# see /usr/share/doc/bash/examples/startup-files (in the package bash-doc)
# for examples
# If not running interactively, don't do anything
case $- in
    *i*) ;;
      *)
      export PATH=$PATH:/usr/local/cuda-12.2/bin
      export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-12.2/lib64
      export ARM_COMPUTELIB=$ARM_COMPUTELIB:/usr/local/arm_compute
      return;;
esac

Update the paths as:

PATH="/usr/local/cuda-12.2/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin"
LANG="en_US.UTF-8"
LD_LIBRARY_PATH="/usr/local/cuda-12.2/lib64"

Source the .bashrc file to apply the chnages, reboot the device and update the MATLAB script:

setenv('PATH', [getenv('PATH') ':/usr/local/cuda-12.2/bin']);
setenv('LD_LIBRARY_PATH', [getenv('LD_LIBRARY_PATH') ':/usr/local/cuda-12.2/lib64']);
coder.checkGpuInstall(envCfg);

By updating the environment variables to point to the correct CUDA version directory, MATLAB should be able to find nvcc and proceed with the GPU code generation process.

[Ryan]

Thank you for your answer. It is true that the result of "which nvcc" does give /usr/local/cuda-12.2. However, the /usr/local/cuda folder is linked to this folder so I'm not sure this is the root of the issue.