并行程序设计导论 并行程序设计导论 (Peter Pacheco)（Z-Library）

【One-Line Pitch】 A practical, bottom-up introduction to parallel programming that teaches you how to write efficient distributed-memory programs with MPI and shared-memory programs with Pthreads and OpenMP, ideal for undergraduate CS students and self-taught developers who want to move from sequential to parallel thinking. 【Book Arc】 - **Opening (~0%–15%)**: The book sets the stage by explaining why parallel computing matters—why we need it, what hardware trends (multi-core, clusters) make it possible, and the fundamental concepts of concurrency, parallelism, and performance limits like Amdahl's Law. This stage builds the mental model you'll need for everything that follows. - **Early (~15%–40%)**: It dives into distributed-memory programming with MPI, covering the core communication model (message passing), point-to-point operations (send/receive), and collective operations (broadcast, reduce, gather). The focus is on getting you to write working parallel programs that split data across processes and coordinate via messages. - **Middle (~40%–65%)**: The book shifts to shared-memory programming with Pthreads, introducing thread creation, synchronization primitives (mutexes, condition variables), and the classic pitfalls like race conditions and deadlock. This stage emphasizes correctness—how to protect shared data and reason about thread interleavings. - **Late (~65%–85%)**: It covers OpenMP as a higher-level, directive-based alternative for shared-memory programming, showing how to parallelize loops and regions with minimal code changes. This stage contrasts the explicit control of Pthreads with the convenience of OpenMP, and discusses when to choose which. - **Ending (~85%–100%)**: The book wraps up with practical considerations—performance evaluation, debugging parallel programs, and a discussion of broader parallel architectures and algorithms. It ties the three programming models together and points toward more advanced topics, with programming exercises of varying difficulty throughout to consolidate learning. 【Key Takeaways】 - **Parallelism is a hardware reality, not an option** (Early): Multi-core processors and clusters mean that sequential programs leave performance on the table; understanding the hardware landscape (memory hierarchies, network latency) is the prerequisite for writing effective parallel code. - **MPI is the standard for distributed memory** (Early): Message passing is the core abstraction—processes have private memory and coordinate by sending/receiving data. Mastery of point-to-point and collective operations lets you scale programs across machines, not just cores. - **Race conditions are the #1 enemy in shared memory** (Middle): When threads share data, unsynchronized access leads to nondeterministic bugs. The book drills in the discipline of using mutexes and condition variables to enforce mutual exclusion and ordering. - **Pthreads give you control, OpenMP gives you productivity** (Late): Pthreads require explicit thread management and are error-prone but flexible; OpenMP lets you annotate sequential code with pragmas to parallelize loops quickly, trading fine-grained control for ease of use. - **Performance is measured, not assumed** (Ending): Speedup and efficiency are the key metrics, and Amdahl's Law reminds you that the serial fraction of your code caps your gains. Profiling and careful benchmarking are essential to know if your parallelization actually pays off. - **Debugging parallel programs is qualitatively harder** (Ending): Bugs like deadlocks and data races are intermittent and hard to reproduce; the book emphasizes systematic testing, logging, and tools to catch these issues early. - **The three models are complementary, not competing** (Ending): MPI for distributed systems, Pthreads for low-level shared memory, OpenMP for quick wins on multicore—choosing the right tool depends on your hardware target and performance needs. 【Reading Tips】 - **Skim the hardware overview in the opening** if you already know why multi-core exists; the key takeaway is Amdahl's Law and the memory model, which you'll need later. - **Deep-read the MPI chapters** and actually code the examples—message passing is best learned by doing. Focus on the difference between blocking and non-blocking communication, as that's a common source of confusion. - **For Pthreads, pay extra attention to the synchronization sections**; race conditions and deadlock are subtle. Work through the exercises that ask you to modify shared data structures—they'll cement the concepts. - **Treat OpenMP as a "quick win" chapter**—you can skim the directive syntax and focus on the pragmas for parallel loops and reductions, then compare the code complexity with the Pthreads version of the same problem. - **Use the end-of-chapter exercises as a self-test**; they range from easy to hard, so start with the simple ones to build confidence, then attempt the harder ones to stretch your understanding. 【Coverage Limits】 The excerpts cover the book's overall scope and structure but do not include detailed code examples, specific exercise content, or the full treatment of advanced topics like GPU programming or hybrid MPI+OpenMP models. This guide synthesizes the book's arc and key concepts from the blurb and general knowledge of the subject matter.