Abstract:

This text provides a comprehensive, multi-chapter introduction to the Kokkos C++ programming model, which is designed for writing performance-portable code across diverse hardware architectures like CPUs and GPUs. The document begins by establishing the core principles of Kokkos, including its use of descriptive parallel patterns (parallel_for, parallel_reduce), and the fundamental concepts of execution spaces (where code runs) and memory spaces (where data lives).

It then delves into Kokkos's primary data management tool, the Kokkos::View, a sophisticated multidimensional array class that abstracts memory layout and automates memory management. Techniques for managing data across different memory spaces, such as mirror views for data transfer and dual views for synchronized data, are also detailed. Finally, the text explores advanced parallelism techniques, introducing hierarchical parallelism through thread teams to exploit nested levels of concurrency. This includes thread-level (TeamThreadRange), vector-level (ThreadVectorRange), and team-local memory (scratch pads). The document concludes with an overview of Kokkos's task-based programming model for handling computations with complex dependencies.

A Guide to Performance Portability with Kokkos

• 24.1 The Kokkos Model: Kokkos is a C++ programming model aimed at performance portability, allowing a single source code to run efficiently on various hardware architectures (CPUs, GPUs). It is descriptive, meaning programmers define the parallel structure of their computation, and Kokkos maps it to the target hardware.
• 24.2 Basic Parallel Patterns: At its core, Kokkos executes parallel operations composed of three parts: a parallel pattern (e.g., Kokkos::parallel_for, Kokkos::parallel_reduce), an execution policy that defines how to iterate, and a computational body (a functor or, preferably, a KOKKOS_LAMBDA) that contains the code to be executed.
• 24.3 Execution Spaces: An execution space defines where parallel code runs (e.g., on CPU threads via OpenMP or on an NVIDIA GPU via CUDA). This can be explicitly specified in the execution policy to control the target hardware for a given parallel operation.
• 24.4 Memory Spaces: A memory space is where data is stored, such as CPU RAM or GPU RAM. Kokkos provides portable memory allocation functions (Kokkos::kokkos_malloc, Kokkos::kokkos_free) that automatically allocate data in the memory space corresponding to the default execution space.
• 25.1 Kokkos Views: Views (Kokkos::View) are the preferred method for memory management in Kokkos. They are smart, multidimensional array-like objects that automatically handle memory allocation, deallocation, and data layout, abstracting away the complexity of managing memory on different hardware.
• 25.1.1 Data Layouts: Views can be configured with different memory layouts to optimize access patterns. Kokkos::LayoutRight (row-major) is ideal for CPU caches, while Kokkos::LayoutLeft (column-major) is better for coalesced memory access on GPUs.
• 25.1.2 Memory Traits: Views can be customized with memory traits to define specific behaviors. Key traits include Kokkos::Unmanaged (to wrap existing, non-Kokkos memory), and Kokkos::Atomic (to enforce that the view is only accessed via atomic operations).
• 25.2 Mirrors and Dual Views: To manage data between the host (CPU) and a device (GPU), Kokkos provides mirror views (create_mirror_view) for creating copies and Kokkos::deep_copy for explicit data transfer. For more complex scenarios where data is actively used in both spaces, a Kokkos::DualView can be used to manage synchronization.
• 26.1 Hierarchical Parallelism: Kokkos supports expressing parallelism at multiple levels, which is essential for modern hardware. This is achieved by organizing threads into teams.
• 26.1.1 Thread Teams: The foundation for hierarchical parallelism is the Kokkos::TeamPolicy. It organizes threads into a two-dimensional structure: a "league" of teams, where each team contains a set of threads that can cooperate and synchronize.
• 26.1.2 & 26.1.3 Team and Vector Parallelism: Inside a kernel launched with a TeamPolicy, further parallelism can be dispatched. Kokkos::TeamThreadRange distributes loop iterations across threads within a team, while Kokkos::ThreadVectorRange is used for loops that can be vectorized using SIMD instructions.
• 26.1.4 Scratch Pads: A scratch pad is a small, fast, team-local memory space, analogous to shared memory in CUDA. It is used to reduce reliance on slower global memory by allowing threads within a team to share data efficiently.
• 26.2 Tasking: Kokkos provides a task-based programming model for problems that consist of many small tasks with arbitrary dependencies. This involves defining task functors, which are managed and executed by a scheduler, offering an alternative way to express complex parallel workflows.