Page 1

(This page has no text content)

Page 2

Mastering SIMD with Java Vector API Unlocking Single-Core Performance Through SIMD Optimization Roman Snytsar

Page 3

Mastering SIMD with Java Vector API: Unlocking Single-Core Performance Through SIMD Optimization ISBN-13 (pbk): 979-8-8688-2675-7 ISBN-13 (electronic): 979-8-8688-2676-4 https://doi.org/10.1007/979-8-8688-2676-4 Copyright © 2026 by Roman Snytsar This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Trademarked names, logos, and images may appear in this book. Rather than use a trademark symbol with every occurrence of a trademarked name, logo, or image we use the names, logos, and images only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein. Managing Director, Apress Media LLC: Welmoed Spahr Acquisitions Editor: Melissa Duffy Editorial Assistant: Gryffin Winkler Cover designed by eStudioCalamar Cover image designed by Pexels Distributed to the book trade worldwide by Springer Science+Business Media New York, 1 New York Plaza, New York, NY 10004. Phone 1-800-SPRINGER, fax (201) 348-4505, e-mail orders-ny@springer-sbm.com, or visit www.springeronline.com. Apress Media, LLC is a Delaware LLC and the sole member (owner) is Springer Science + Business Media Finance Inc (SSBM Finance Inc). SSBM Finance Inc is a Delaware corporation. For information on translations, please e-mail booktranslations@springernature.com; for reprint, paperback, or audio rights, please e-mail bookpermissions@springernature.com. Apress titles may be purchased in bulk for academic, corporate, or promotional use. eBook versions and licenses are also available for most titles. For more information, reference our Print and eBook Bulk Sales web page at http://www.apress.com/bulk-sales. Any source code or other supplementary material referenced by the author in this book is available to readers on GitHub. For more detailed information, please visit https://www.apress.com/gp/services/ source-code. If disposing of this product, please recycle the paper Roman Snytsar Sammamish, WA, USA

Page 4

To my wife: my first reader, my best critic, and my biggest fan.

Page 5

(This page has no text content)

Page 6

(This page has no text content)

Page 7

(This page has no text content)

Page 8

(This page has no text content)

Page 9

(This page has no text content)

Page 10

(This page has no text content)

Page 11

xi About the Author Roman Snytsar is a researcher and software engineer with 20 years of experience at Microsoft Research. His work spans performance optimization, computer science theory, and practical algorithm implementation. He has published multiple patents and articles focused on explaining complex technical concepts in accessible ways. With deep expertise in both theoretical foundations and real-world performance- critical code, Roman brings a unique perspective to making advanced programming techniques accessible to working developers. His research background, combined with hands- on engineering experience, makes him uniquely qualified to bridge the gap between academic SIMD theory and practical Java development.

Page 12

xiii About the Technical Reviewer Daniel Lemire is a computer science professor at the Université du Québec (TÉLUQ), where he specializes in high-performance programming, software optimization, and data indexing techniques, with a particular emphasis on vectorization and SIMD instructions to accelerate computational tasks. Ranked among the top 2% of scientists worldwide by Stanford University and Elsevier, he has authored over 100 peer-reviewed papers and contributed performance-enhancing code to major libraries and web browsers, including innovations in JSON parsing that earned him the University of Quebec’s 2020 Award of Excellence in Research. As one of GitHub's top 1000 developers with over 100 million users on the platform, Lemire frequently explores vector programming in languages through his influential blog and open source projects.

Page 13

xv Introduction Speed changes everything. A program that runs ten times faster is not merely more convenient—it becomes capable of tasks previously impossible. Real-time becomes truly real. Overnight batch jobs finish before lunch. Simulations that once demanded clusters now run on laptops. This book is about achieving that kind of speed: not through exotic hardware or architectural overhauls, but by unlocking the vector processing power that already lies dormant in your CPU. Modern processors can execute the same operation on 8, 16, or even 64 data elements simultaneously—yet most Java code touches none of this capability. You will learn to change that. By mastering the techniques in these pages, you will write programs that fully exploit the silicon you already own, transforming performance from a constraint into a competitive advantage. The Parallelism Hierarchy Modern applications—from real-time analytics to machine learning and scientific modeling—demand ever-increasing computational performance. To meet these demands, developers of complex systems must exploit several hierarchical levels of parallelism, ranging from distributed clusters down to parallel acceleration within a single processor core. At the highest level, parallel processing algorithms distribute computations across multiple systems or nodes, often organized into clusters or deployed in cloud environments. This model proves essential for analyzing large datasets and supporting fault-tolerant, scalable architectures. Platforms like Hadoop and Spark abstract away the complexity of distributed systems, enabling developers to implement parallel solutions through various abstractions: the map-reduce paradigm, resilient distributed datasets (RDD), and dataflow graphs. These platforms handle task scheduling, data partitioning, and inter-node communication, freeing developers to concentrate on application logic. Within a single system, multi-threading enables parallel code execution across multiple CPU cores. Application and library developers readily access this model through high-level languages. Multi-threading works well for decomposing tasks into independent units, such as processing multiple client requests, parallelizing

Page 14

xvi loops, or performing asynchronous I/O. Java developers use constructs like Thread, ExecutorService, ForkJoinPool, and parallel streams to express multi-threaded parallelism. Despite the prevalence of distributed and thread-level models, real-world performance often hinges on single-core execution efficiency. This level of parallelism is less visible to most developers, but is crucial for optimizing compute-intensive workloads. Acceleration is achieved through two mechanisms: • Data-level parallelism (DLP) allows a single core to process multiple data elements simultaneously using vector registers. This is the domain of SIMD (Single Instruction, Multiple Data) operations, where one instruction operates on a vector of values. • Instruction-level parallelism (ILP) allows multiple machine instructions to execute simultaneously within a single core through pipelining, superscalar execution, and out-of-order execution. Hardware implements these techniques to boost performance substantially, yet they remain invisible to most developers. Traditionally, exploiting single-core parallelism required low-level programming in assembly language or using C/C++ intrinsics—cumbersome, error-prone, and difficult to maintain. This additional complexity creates a barrier that limits most Java developers' access to performance-critical optimizations. What This Book Is About The Java Vector API changes this equation. Introduced as an incubating feature in JDK 16 and progressing toward standardization, it provides a platform-independent abstraction over SIMD instructions. For the first time, Java developers can explicitly express vector operations without leaving the safety and portability of the JVM ecosystem. The Vector API opens data-level parallelism to a broader range of developers by • Providing a platform-independent abstraction over SIMD instructions • Using familiar object-oriented Java constructs to express vector operations • Enabling both automatic vectorization and explicit vector programming in the JVM InTroduCTIon

Page 15

xvii With the Vector API, developers can write high-performance code that leverages hardware acceleration without leaving the Java ecosystem. This fundamentally changes performance optimization in Java, especially in domains like finance, scientific computing, and machine learning. Yet data-level parallelism tells only half the story. This book also explores instruction-level parallelism: the art of structuring code so that multiple operations execute simultaneously within a single core. Through techniques like loop unrolling, asymmetric loop construction, and careful attention to structural hazards, we learn to exploit the full capabilities of modern superscalar processors. The interplay between SIMD vectorization and instruction-level optimization often determines whether a solution achieves modest gains or dramatic speedups. Mastering these techniques, however, requires more than memorizing APIs and patterns. The deeper challenge lies in recognizing which problems yield to vectorization, understanding why certain approaches succeed while others fail, and developing the intuition to make sound performance decisions. This book addresses all three. A Different Way of Thinking While Java developers are well-versed in high-level parallelism abstractions, exploiting single-core parallelism requires fundamentally different thinking. Unlocking the full potential of the Vector API demands a paradigm shift in how developers reason about performance, data, and control flow. This book invites you on a journey through familiar problems, viewed with fresh eyes. Many of the problems examined here are known to anyone who has prepared for technical interviews or studied computer science fundamentals. These problems were deliberately chosen for their simplicity and applicability. However, this book's goal is not to rehash well-worn solutions. Instead, we present a new way of thinking about these problems: one based on low-level parallelism, with particular emphasis on data-level and instruction-level optimization. By viewing these problems through the lens of vectorization, we discover acceleration opportunities often hidden beneath layers of abstraction. We invite readers to move beyond traditional algorithmic reasoning and embrace a performance-oriented approach: • Recognizing program patterns that lend themselves to parallel execution InTroduCTIon

Page 16

xviii • Understanding how modern processors execute code at the microarchitectural level • Learning to explicitly express parallelism using the Java Vector API The book's ultimate goal is to equip developers with the intuition and practical skills needed to apply this knowledge to real-world software projects. Whether optimizing a numerical kernel, accelerating a data pipeline, or improving the responsiveness of a user-facing application, the techniques covered here can help push systems to new levels of efficiency and scalability. By bridging the gap between algorithmic clarity and hardware-oriented optimization, this book hopes to inspire a new generation of developers to write code that is not only correct and elegant but also fast. Who This Book Is For This book is written for Java developers who want to push the performance boundaries of their applications. You should be comfortable with Java syntax and have some familiarity with algorithms and data structures. Prior experience with low-level optimization or assembly language is not required—indeed, one of this book's goals is to make these concepts accessible without requiring such background. You will benefit most from this book if you • Work on performance-critical applications in domains such as scientific computing, financial analysis, machine learning, or data processing • Want to understand how modern processors execute code at the microarchitectural level • Seek practical techniques that can be applied immediately to real- world problems • Appreciate both the elegance of algorithms and the pragmatism of benchmarks This book is not an exhaustive API reference. The official documentation serves that purpose admirably. Instead, we focus on the reasoning behind vectorization decisions, the patterns that recur across diverse problems, and the pitfalls that await the unwary. InTroduCTIon

Page 17

xix How This Book Is Organized Rather than grouping all chapters of similar type together, we have deliberately interspersed three thematic threads throughout the book: foundational techniques that build your vector programming vocabulary, classic algorithmic problems reimagined through the lens of parallelism, and mathematical explorations drawn from Project Euler. This arrangement serves a practical purpose: after grappling with abstract concepts like prefix sums or horizontal operations, you encounter a concrete problem that puts those concepts to work. The alternation between theory and application, between building blocks and complete solutions, aims to keep the material engaging while reinforcing understanding through varied repetition. The threads are loosely categorized below, though in practice, they weave together throughout the chapters. Thread I: Foundations Chapter 1: Programming Vector Hardware surveys the landscape of vector-capable processors. We examine x86 architectures (SSE, AVX, AVX-512), ARM implementations (NEON, SVE), and the emerging RISC-V vector extension. The chapter introduces the concept of vector species and explains how to write code that adapts gracefully to different hardware capabilities. We also describe the diverse test platforms used throughout this book. Chapter 2: Vector Addition begins with the deceptively simple problem of summing an array. This chapter introduces lanewise operations, explores the subtleties of floating- point arithmetic under IEEE 754, and presents Kahan's compensated summation algorithm as a remedy for rounding errors. Chapter 3: A Taste of Statistics applies our growing toolkit to computing mean and standard deviation. The chapter introduces the crucial distinction between horizontal and vertical vector operations, explains cache-oblivious algorithms, and demonstrates how to minimize expensive horizontal operations by restructuring computations. Chapter 6: Prefix Sum tackles the fundamental building block of many parallel algorithms. We examine the tension between theoretical elegance and practical performance, discovering that vector scan operations face inherent challenges from sequential dependencies. InTroduCTIon

Page 18

xx Chapter 9: Random Number Generator reveals the mathematical structure behind java.util.Random and shows how to vectorize the Linear Congruential Generator. The chapter uses the prefix sum algorithm and demonstrates the compress operation for rejection sampling. Thread II: Classic Problems Reimagined Chapter 5: Removing Duplicates introduces the powerful compress operation, which selectively writes vector elements based on a mask. We compare multiple implementation strategies, including the classic two-pointer method and its vectorized counterparts. Chapter 7: Finding the Unique Element explores bit manipulation and mask operations through problems involving XOR reduction. The chapter introduces trit arithmetic and examines how branch misprediction influences performance decisions. Chapter 10: Trapping Rainwater showcases the most sophisticated optimization journey in the book. Starting from a standard interview solution, we progress through vector scans, Blelloch's work-optimal algorithm, vector transposition, and memoization. This chapter demonstrates how multiple techniques combine to achieve substantial speedups. Chapter 12: Merging Arrays presents the merge path algorithm and bitonic sorting—techniques that transform the familiar merge operation into a parallel-friendly form. The chapter also discusses structural hazards and demonstrates an asymmetric loop optimization that improves instruction-level parallelism. Chapter 14: Maximum Subarray Sum revisits Kadane's celebrated algorithm through the lens of vectorization. We reformulate the problem using prefix sums, revealing connections between seemingly different algorithmic approaches. Thread III: Project Euler Explorations The final thread applies our techniques to problems from Project Euler, a collection of mathematical puzzles that has challenged programmers since 2001. These chapters demonstrate that vectorization can illuminate even well-studied problems. Chapter 4: Multiples of Three or Five (Project Euler 1) provides a deep dive into JIT compilation, examining the assembly code generated by the C1 and C2 compilers. We encounter Lemire's divisibility test and explore how masked operations enable branch- free conditional logic. InTroduCTIon

Page 19

xxi Chapter 8: Even Fibonacci Numbers (Project Euler 2) introduces matrix exponentiation as a vectorization strategy and contrasts Array of Structures with Structure of Arrays data layouts. Chapter 11: Convergents of e (Project Euler 65) ventures into arbitrary-precision arithmetic, applying Strassen's and Karatsuba's algorithms to accelerate BigInteger operations. The chapter introduces the creative "billionaire" algorithm using a gigenary number system. Chapter 13: Largest Product in a Series (Project Euler 8) develops a novel sliding window algorithm using suffix and prefix products. The chapter introduces the gather operation for indexed memory access. Chapter 15: The Collatz Conjecture (Project Euler 14) confronts the challenge of divergent execution paths. We introduce the expand operation, work piling strategies, and comprehensive memoization. This chapter also delivers a cautionary tale about over-optimization. Reading the Code The code in this book prioritizes clarity of vectorization concepts over production-ready style. You will notice: • Minimal abstraction: Methods are often long and repetitive rather than factored into helper functions. This is deliberate. Function call overhead, while usually negligible, can obscure performance characteristics in microbenchmarks. More importantly, seeing the complete algorithm in one place aids comprehension. • Explicit unrolling: Many algorithms appear in both basic and unrolled variants. Unrolling exposes instruction-level parallelism opportunities that the JIT compiler might otherwise miss. • Platform-specific variants: Some chapters include implementations optimized for specific vector lengths. While the book emphasizes vector-length-agnostic programming, there are occasions when targeting a known platform yields worthwhile gains. All source code is available in the accompanying repository. We encourage you to run the benchmarks on your own hardware—results may differ from those reported here, and understanding why is itself instructive. InTroduCTIon

Page 20

xxii What This Book Does Not Cover To maintain focus, we have deliberately excluded several related topics: • GPU programming: While GPUs excel at data-parallel workloads, they require different APIs and programming models. The Java Vector API targets CPU vector units. • Automatic vectorization: Modern JIT compilers can sometimes vectorize scalar loops automatically. We focus on explicit vectorization, where the programmer controls the transformation. • Multi-threading: Combining SIMD with multi-core parallelism offers additional speedups, but introduces complexity orthogonal to our focus. Each chapter's benchmarks measure single-core performance. • API completeness: We cover the operations most relevant to our examples rather than exhaustively documenting every method. The official Javadoc remains the authoritative reference. Getting Started To run the examples in this book, you will need • JDK 25 or later (the Vector API remains in incubator/preview status; check current documentation for the appropriate module flags) • A processor with vector capabilities (virtually all modern x86 and ARM processors qualify) • Familiarity with your IDE's configuration for enabling preview features The journey ahead will challenge your assumptions about what constitutes "fast" code. Problems you thought you understood will reveal hidden depths. Algorithms you considered optimal will yield to parallel reformulation. And occasionally, despite your best efforts, the scalar baseline will win—a humbling reminder that optimization is empirical, not ideological. InTroduCTIon

Tip the Site

Mastering SIMD with Java Vector API Unlocking Single-Core Performance Through SIMD Optimization (Roman Snytsar)（Z-Library）

Mastering SIMD with Java Vector API Unlocking Single-Core Performance Through SIMD Optimization (Roman Snytsar)（Z-Library）

Registered users can read the full content for free

AI Reading Assistant

Passage locations

Recommended for You

Tip the Site