CUDA C编程权威指南 (高性能计算技术丛书) (程润伟（John Cheng）)（Z-Library）

【One-Line Pitch】 A comprehensive, hands-on guide to CUDA C programming that takes you from the fundamentals of heterogeneous computing to advanced multi-GPU optimization, ideal for developers and researchers who want to harness NVIDIA GPUs for high-performance parallel computing. 【Book Arc】 - **Opening (~0%–10%)**: Introduces the concept of heterogeneous computing, contrasting CPU and GPU architectures, and explains why CUDA is the platform of choice for parallel programming. Sets up the development environment with a simple "Hello World" example. - **Early (~10%–35%)**: Dives into the CUDA programming model—memory management, thread organization, kernel launching—and then the execution model, covering warp behavior, occupancy, and branch divergence. Includes practical techniques like loop unrolling and reduction patterns. - **Middle (~35%–65%)**: Explores the memory hierarchy in depth: global memory access patterns, shared memory and constant memory optimization, and warp shuffle instructions. Introduces streams and events for concurrent kernel execution and overlapping data transfers. - **Late (~65%–90%)**: Covers instruction-level primitives (float vs. double, intrinsics, atomics), GPU-accelerated libraries (cuSPARSE, cuBLAS, cuFFT, cuRAND), and OpenACC for directive-based programming. Moves into multi-GPU programming, including peer-to-peer communication and MPI integration. - **Ending (~90%–100%)**: Focuses on the full development cycle—profiling with nvprof/nvvp, debugging kernels and memory, and a case study porting a C program (crypt) to CUDA C, from evaluation to deployment. 【Key Takeaways】 - **Heterogeneous computing is the foundation** (Early): CUDA combines CPU (host) and GPU (device) strengths, with the CPU managing resources and the GPU accelerating compute-intensive parallel sections—understand this division before writing any kernel. - **The CUDA programming model is logical, not physical** (Early): You organize threads into blocks and grids, and the hardware schedules them on streaming multiprocessors; mastering this abstraction is key to writing correct and scalable kernels. - **Memory access patterns determine performance** (Middle): Global memory performance hinges on aligned and coalesced access; restructuring data from arrays of structures (AoS) to structures of arrays (SoA) can dramatically improve bandwidth. - **Shared memory is your manual cache** (Middle): Using shared memory to reduce global memory traffic—like in matrix transpose or reduction kernels—is one of the highest-impact optimizations, especially when combined with padding to avoid bank conflicts. - **Warp-level thinking is essential** (Middle): Threads execute in warps of 32; branch divergence and resource allocation directly affect occupancy and latency hiding, so design kernels to keep warps uniform and occupancy high. - **Streams enable concurrency** (Late): CUDA streams and events allow overlapping kernel execution with data transfers and CPU work, unlocking grid-level parallelism that is critical for real-world applications. - **Libraries and directives save time** (Late): cuBLAS, cuFFT, and cuRAND provide battle-tested implementations, while OpenACC offers a lower-entry barrier for porting existing code—use them before hand-tuning kernels. - **Profiling drives optimization** (Ending): The APOD cycle (Assess, Parallelize, Optimize, Deploy) with nvprof and nvvp is the systematic way to find bottlenecks and verify improvements, not guesswork. 【Reading Tips】 - **Skim Chapter 1** if you already know GPU basics; it's a gentle intro, but the "Hello World" example is worth running to verify your toolchain. - **Deep-read Chapters 2–3** for the programming and execution models—these are the conceptual core; work through the matrix indexing and reduction examples to internalize thread organization and warp behavior. - **Chapter 5 (shared memory) is the optimization heart**—study the matrix transpose and reduction kernels carefully; the padding and loop-unrolling variations are classic patterns you'll reuse. - **Chapters 8–9 are reference-heavy**: skim the library APIs (cuSPARSE, cuBLAS, etc.) and focus on the multi-GPU patterns (peer-to-peer, MPI) only if your project needs them. - **Don't skip the profiling sections** (Chapters 3 and 10): learning nvprof/nvvp early will save you hours of guesswork later; the crypt case study in Chapter 10 ties everything together. 【Coverage Limits】 The excerpts are primarily table-of-contents and front-matter; detailed code examples and specific performance numbers are not covered in this guide. The guide reflects the book's structure and key themes, not its full technical depth.