Installing Nvidia CUDA and cuDNN on Debian 12 Bookworm

Learn how to install NVIDIA CUDA and cuDNN on Debian 12 Bookworm, optimizing your system for GPU-accelerated computing.

Debian 12 "Bookworm" is the latest version of the popular Linux distribution, known for its stability and extensive software repository. If you're a developer or researcher working with machine learning or deep learning tasks, installing NVIDIA CUDA and cuDNN on your Debian system can significantly enhance performance. This guide will walk you through the step-by-step process of installing these essential components, ensuring that your system is optimized for GPU-accelerated computing.

Understanding NVIDIA CUDA and cuDNN

Before we dive into the installation process, let's briefly understand what NVIDIA CUDA and cuDNN are.

NVIDIA CUDA (Compute Unified Device Architecture) is a parallel computing platform that allows developers to harness the power of GPUs for general-purpose computation. It provides an API (Application Programming Interface) along with a runtime environment that enables developers to write programs using high-level languages like C, C++, or Python while leveraging the computational capabilities of NVIDIA GPUs.

cuDNN (CUDA Deep Neural Network Library) is a GPU-accelerated library specifically designed for deep neural networks. It provides highly optimized implementations of key building blocks such as convolutions, pooling operations, activation functions, etc., which are crucial in accelerating deep learning workloads.

Checking System Requirements

Before proceeding with the installation process, ensure that your system meets the necessary requirements:

• A compatible NVIDIA GPU.
• A supported version of Debian 12 "Bookworm".
• An internet connection to download necessary packages.

Installing Dependencies 

To begin with, we need to install some prerequisite packages required by both CUDA and cuDNN libraries:

Open up your terminal and execute the following commands:

sudo apt update
sudo apt upgrade
sudo apt install build-essential

These commands will update your package list, upgrade existing packages, and install essential build tools required for compilation.

Installing NVIDIA CUDA

Now let's proceed with the installation of NVIDIA CUDA:

• Step 1: Download the latest version of NVIDIA CUDA toolkit from the official NVIDIA website (https://developer.nvidia.com/cuda-downloads).

wget <CUDA Toolkit URL>

Replace <CUDA Toolkit URL> with the actual download link provided on the website.

• Step 2: Once downloaded, navigate to the directory where you saved the installer file and make it executable:

chmod +x cuda_<version>_linux.run

Replace <version> with the appropriate version number in your case.

• Step 3: Run the installer and follow the on-screen instructions:

sudo ./cuda_<version>_linux.run

During installation, you will be prompted to accept license terms, choose installation options, and set up environment variables. Make sure to carefully read each prompt before proceeding.

• Step 4: After successful installation, update your system's PATH variable by adding CUDA binaries location to it. Open ~/.bashrc file using a text editor:

nano ~/.bashrc

Add this line at the end of the file:

export PATH=/usr/local/cuda/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
source ~/.bashrc

Save and exit (Ctrl+X, then Y, followed by Enter).

• 5. Verifying CUDA Installation To verify that CUDA is installed correctly on your Debian system, run these commands in your terminal:

nvcc --version

nvidia-smi

The first command should display information about your installed CUDA version without any errors. The second command will show details regarding your GPU device if it is correctly recognized by the system.

Installing cuDNN

Now that we have CUDA installed, let's proceed with installing cuDNN:

• Step 1: Visit the NVIDIA Developer website (https://developer.nvidia.com/rdp/cudnn-download) and download the appropriate version of cuDNN for your CUDA installation.
• Step 2: Extract the downloaded package using the following command:

tar -xzvf <cuDNN Archive>.tgz

Replace <cuDNN Archive> with the actual filename you downloaded.

• Step 3: Copy the necessary files to their respective locations:

sudo cp cuda/include/cudnn*.h /usr/local/cuda/include/
sudo cp cuda/lib64/libcudnn* /usr/local/cuda/lib64/
sudo chmod a+r /usr/local/cuda/include/cudnn*.h
sudo chmod a+r /usr/local/cuda/lib64/libcudnn*

These commands will copy header files and libraries to appropriate directories while ensuring proper permissions are set.

Verifying cuDNN Installation

To verify if cuDNN is installed successfully, compile and run a sample program provided by NVIDIA in their samples directory. Follow these steps:

• Step 1: Navigate to your CUDA samples directory:

cd ~/NVIDIA_CUDA_XYZ_Samples/

Replace XYZ with your specific CUDA version number.

• Step 2: Compile and run one of the sample programs, for example:

make -C 0_Simple/vectorAdd

./bin/x86_64/linux/release/vectorAdd

If everything is set up correctly, you should see output indicating successful execution without any errors or warnings.

Conclusion Congratulations! You have successfully installed NVIDIA CUDA and cuDNN on Debian 12 "Bookworm." Now you can leverage the power of GPU acceleration for your machine learning and deep learning tasks. Remember to keep both CUDA and cuDNN updated to benefit from the latest features, bug fixes, and performance optimizations.

Remember, installing these libraries correctly is crucial for optimal performance in GPU-based computations. By following this guide, you have equipped yourself with the necessary tools to harness the full potential of NVIDIA GPUs on Debian 12 "Bookworm."