Timestamps are not applicable to a text document. I will omit them.

Abstract:

This article provides a detailed ontology of the NVIDIA CUDA ecosystem to disambiguate its overloaded terminology. It systematically defines core concepts like "CUDA" itself (which has at least five meanings), "driver" (OS-level vs. user-space API), and "kernel" (OS vs. GPU function). The author outlines the layered architecture, distinguishing between the application-facing Runtime API (libcudart) and the system-level Driver API (libcuda), which communicates with the OS kernel driver (nvidia.ko). The article establishes two fundamental conditions for successful application execution: API version compatibility (Driver API version must be ≥ Runtime API version) and GPU code availability (the binary must contain either pre-compiled SASS for the target GPU or portable PTX for JIT compilation). By explaining these rules and detailing what diagnostic tools like nvidia-smi and nvcc actually report, the text serves as a guide for diagnosing version incompatibilities and understanding system behavior. As far as I can tell, it's a correct and useful breakdown of a needlessly confusing system.

A Rigorous Guide to the CUDA Ecosystem

• Overloaded Terminology is the Root Problem: The terms "CUDA," "driver," and "kernel" have multiple, distinct meanings that cause significant confusion. For example, "CUDA" can refer to the programming model, the toolkit, the runtime library, or the GPU instruction set. Precision is required to avoid errors.
• CUDA Has a Layered Architecture: The system is split into a frontend and backend.
  • Frontend (Application Layer): Your application links against the Runtime API (libcudart.so). This provides high-level functions like cudaMalloc.
  • Backend (System Layer): The Runtime API calls the Driver API (libcuda.so), which is a low-level library installed with the GPU driver. This library communicates with the OS kernel driver (nvidia.ko) to manage the hardware.
• Two Core Compatibility Rules: For an application to run, two conditions must be met. Failure to meet them results in specific, common errors.
  1. API Version Match: The system's Driver API version must be greater than or equal to the application's Runtime API version. If not, you get a cudaErrorInsufficientDriver error.
  2. GPU Code Availability: The application binary must contain code the GPU can run. This can be either native SASS (machine code for a specific GPU architecture) or intermediate PTX (bytecode that the driver can Just-In-Time compile for any GPU). If neither is available, you get a cudaErrorNoKernelImageForDevice error.
• Understanding Diagnostic Tools is Crucial: Different tools report different things. Conflating them is a primary source of error.
  • nvidia-smi: Reports the system's GPU driver version and the maximum Driver API version it supports. It knows nothing about your application or which toolkit was used to build it.
  • nvcc --version: Reports the version of the CUDA Toolkit installed and present in the system's PATH. This may not be the toolkit your application was actually compiled with.
  • torch.version.cuda: Reports the version of the CUDA Toolkit that PyTorch was compiled against at build time.
• Driver Compatibility is Forward-Only: A newer driver can run applications built with an older runtime (forward compatibility). An older driver cannot run applications built with a newer runtime (no backward compatibility).
• Practical Advice for Users:
  • Developers: Ship binaries with SASS for common GPU architectures and PTX for forward compatibility with future ones.
  • End-Users: Your main responsibility is ensuring your installed GPU driver is new enough to satisfy the application's runtime requirements. Check this with nvidia-smi.
  • PyTorch/TensorFlow Users: The CUDA version reported by torch.version.cuda dictates the minimum driver version you need. A newer driver is fine.