Optimizing Cloud Native Java Practical Techniques for Improving JVM Application Performance (Benjamin J. Evans, James Gough) (Z-Library)

📄 File Format: PDF

💾 File Size: 16.4 MB

📖 Read Online ⬇️ Download

Registered users can read the full content for free

Register as a Gaohf Library member to read the complete e-book online for free and enjoy a better reading experience.

📄 Page 1

Benjamin J. Evans & James Gough Foreword by Holly Cummins Optimizing Cloud Native Java Practical Techniques for Improving JVM Application Performance

📄 Page 2

9 7 8 1 0 9 8 1 4 9 3 4 5 5 6 9 9 9 ISBN: 978-1-098-14934-5 US $69.99 CAN $87.99 JAVA PROGR AMMING Performance tuning is an experimental science, but that doesn’t mean engineers should resort to guesswork and folklore to get the job done. Yet that’s often the case. With this practical book, intermediate to advanced Java technologists working with complex platforms will learn how to tune Java cloud applications for performance using a quantitative, verifiable, and repeatable approach. In response to the ubiquity of cloud computing, this updated edition of Optimizing Cloud Native Java addresses topics that are key to high performance of Java applications in the cloud. Many resources on performance focus on the theory and internals of Java virtual machines, but this book discusses the low-level technical aspects within the context of performance-tuning practicalities and examines a wide range of aspects. With this book, you will: • Learn how Java principles and technology make the best use of modern hardware, operating systems, and cloud stacks • Examine the pitfalls of measuring Java performance numbers and the drawbacks of microbenchmarking • Understand how to package, deploy, operate, and debug Java/JVM applications in modern cloud environments • Apply emerging observability approaches to obtain deep understanding of cloud native applications Optimizing Cloud Native Java “ Highly recommended for experienced Java engineers and performance practitioners. This book dives deep into garbage collection, observability strategy, and performance tuning of Java applications in the cloud.” Guus Bosman, Distinguished Engineer, Executive Director, Morgan Stanley “A good understanding of Java performance is core to getting the most out of cloud platforms. This book provides practical steps on how to investigate performance problems, as well as advice on how to avoid them in the f irst place.” Elspeth Minty, Managing Director, RBC Capital Markets Benjamin Evans, senior principal software engineer and observability lead at Red Hat Runtimes, is a Java Champion and author of several books, including Optimizing Java and Java in a Nutshell. James Gough is a Distinguished Engineer at Morgan Stanley working on cloud native architecture and API programs. He is a Java Champion and coauthor of Mastering API Architecture.

📄 Page 3

Benjamin J. Evans and James Gough Foreword by Holly Cummins Optimizing Cloud Native Java Practical Techniques for Improving JVM Application Performance SECOND EDITION

📄 Page 4

978-1-098-14934-5 [LSI] Optimizing Cloud Native Java by Benjamin J. Evans and James Gough Copyright © 2025 Benjamin J. Evans and James Gough Ltd. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com. Acquisitions Editor: Brian Guerin Development Editor: Rita Fernando Production Editor: Christopher Faucher Copyeditor: Emily Wydeven Proofreader: Piper Editorial Consulting, LLC Indexer: Potomac Indexing, LLC Interior Designer: David Futato Cover Designer: Karen Montgomery Illustrator: Anna Evans May 2018: First Edition October 2024: Second Edition Revision History for the Second Edition 2024-10-10: First Release See http://oreilly.com/catalog/errata.csp?isbn=9781098149345 for release details. The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Optimizing Cloud Native Java, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc. The views expressed in this work are those of the authors and do not represent the publisher’s views. While the publisher and the authors have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the authors disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights.

📄 Page 5

This book is dedicated to my wife, Anna, who not only illustrated it beautifully but also helped edit portions and, crucially, was often the first person I bounced ideas off. —Ben Evans This book is dedicated to my incredible family Megan, Emily, and Anna. Writing would not have been possible without their help and support. I’d also like to thank my parents, Heather and Paul, for encouraging me to learn and their constant support. I’d also like to thank Ben Evans for his guidance and friendship—it’s been a pleasure working together again. —James Gough

📄 Page 6

(This page has no text content)

📄 Page 7

Table of Contents Foreword. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii 1. Optimization and Performance Defined. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Java Performance the Wrong Way 2 Java Performance Overview 4 Performance as an Experimental Science 5 A Taxonomy for Performance 7 Throughput 7 Latency 8 Capacity 8 Utilization 8 Efficiency 9 Scalability 9 Degradation 9 Correlations Between the Observables 10 Reading Performance Graphs 11 Performance in Cloud Systems 14 Summary 16 2. Performance Testing Methodology. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 Types of Performance Tests 17 Latency Test 18 Throughput Test 19 Stress Test 19 Load Test 20 Endurance Test 20 v

📄 Page 8

Capacity Planning Test 20 Degradation Test 21 Best Practices Primer 22 Top-Down Performance 22 Creating a Test Environment 23 Identifying Performance Requirements 24 Performance Testing as Part of the SDLC 25 Java-Specific Issues 25 Causes of Performance Antipatterns 26 Boredom 27 Résumé Padding 27 Social Pressure 27 Lack of Understanding 28 Misunderstood/Nonexistent Problem 28 Statistics for JVM Performance 29 Types of Error 29 Non-Normal Statistics 33 Interpretation of Statistics 37 Spurious Correlation 38 The Hat/Elephant Problem 41 Cognitive Biases and Performance Testing 44 Reductionist Thinking 45 Confirmation Bias 45 Fog of War (Action Bias) 46 Risk Bias 46 Summary 47 3. Overview of the JVM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 Interpreting and Classloading 49 Executing Bytecode 52 Introducing HotSpot 56 Introducing Just-in-Time Compilation 58 JVM Memory Management 60 Threading and the Java Memory Model 61 Monitoring and Tooling for the JVM 62 Java Implementations, Distributions, and Releases 68 Choosing a Distribution 69 The Java Release Cycle 73 Summary 74 vi | Table of Contents

📄 Page 9

4. Understanding Garbage Collection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 Introducing Mark and Sweep 77 Garbage Collection Glossary 79 Introducing the HotSpot Runtime 80 Representing Objects at Runtime 81 GC Roots 84 Allocation and Lifetime 85 Weak Generational Hypothesis 86 Production GC Techniques in HotSpot 87 Thread-Local Allocation 87 Hemispheric Collection 88 The “Classic” HotSpot Heap 90 The Parallel Collectors 92 Young Parallel Collections 93 Old Parallel Collections 94 Serial and SerialOld 95 Limitations of Parallel Collectors 96 The Role of Allocation 97 Summary 102 5. Advanced Garbage Collection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 Tradeoffs and Pluggable Collectors 105 Concurrent GC Theory 107 JVM Safepoints 108 Tri-Color Marking 110 Forwarding Pointers 113 G1 114 G1 Heap Layout and Regions 115 G1 Collections 117 G1 Mixed Collections 118 Remembered Sets 119 Full Collections 120 JVM Config Flags for G1 121 Shenandoah 122 Concurrent Evacuation 123 JVM Config Flags for Shenandoah 123 Shenandoah’s Evolution 124 ZGC 125 Balanced (Eclipse OpenJ9) 128 OpenJ9 Object Headers 129 Large Arrays in Balanced 130 Table of Contents | vii

📄 Page 10

NUMA and Balanced 131 Niche HotSpot Collectors 132 CMS 132 Epsilon 134 Summary 134 6. Code Execution on the JVM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 Lifecycle of a Traditional Java Application 138 Overview of Bytecode Interpretation 140 Introduction to JVM Bytecode 142 Simple Interpreters 149 HotSpot-Specific Details 150 JIT Compilation in HotSpot 152 Profile-Guided Optimization 152 Klass Words, Vtables, and Pointer Swizzling 154 Compilers Within HotSpot 156 The Code Cache 157 Logging JIT Compilation 159 Simple JIT Tuning 161 Evolving Java Program Execution 162 Ahead-of-Time (AOT) Compilation 163 Quarkus 164 GraalVM 167 Summary 168 7. Hardware and Operating Systems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 Introduction to Modern Hardware 170 Memory 171 Memory Caches 172 Modern Processor Features 177 Translation Lookaside Buffer 177 Branch Prediction and Speculative Execution 177 Hardware Memory Models 178 Operating Systems 179 The Scheduler 180 The JVM and the Operating System 182 Context Switches 183 A Simple System Model 184 Utilizing the CPU 186 Garbage Collection 188 I/O 189 viii | Table of Contents

📄 Page 11

Mechanical Sympathy 190 Summary 191 8. Components of the Cloud Stack. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 Java Standards for the Cloud Stack 194 Cloud Native Computing Foundation 195 Kubernetes 197 Prometheus 197 OpenTelemetry 197 Virtualization 198 Choosing the Right Virtual Machines 200 Virtualization Considerations 201 Images and Containers 202 Image Structure 202 Building Images 203 Running Containers 205 Networking 205 Introducing the Fighting Animals Example 206 Summary 208 9. Deploying Java in the Cloud. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211 Working Locally with Containers 212 Docker Compose 212 Tilt 214 Container Orchestration 215 Deployments 216 Sharing Within Pods 217 Container and Pod Lifecycles 218 Services 219 Connecting to Services on the Cluster 220 Challenges with Containers and Scheduling 221 Working with Remote Containers Using “Remocal” Development 223 Deployment Techniques 223 Blue/Green Deployments 224 Canary Deployments 226 Evolutionary Architecture and Feature Flagging 229 Java-Specific Concerns 232 Containers and GC 232 Memory and OOMEs 233 Summary 234 Table of Contents | ix

📄 Page 12

10. Introduction to Observability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235 The What and the Why of Observability 235 What Is Observability? 236 Why Observability? 237 The Three Pillars 238 Metrics 238 Logs 241 Traces 243 The Pillars as Data Sources 247 Profiling—A Fourth Pillar? 248 Observability Architecture Patterns and Antipatterns 249 Architectural Patterns for Metrics 249 Manual Versus Automatic Instrumentation 250 Antipattern: Shoehorn Data into Metrics 252 Antipattern: Abusing Correlated Logs 253 Diagnosing Application Problems Using Observability 254 Performance Regressions 254 Unstable Components 256 Repartitioning and “Split-Brain” 256 Thundering Herd 258 Cascading Failure 259 Compound Failures 260 Vendor or OSS Solution? 261 Summary 263 11. Implementing Observability in Java. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265 Introducing Micrometer 266 Meters and Registries 266 Counters 268 Gauges 269 Meter Filters 271 Timers 273 Distribution Summaries 274 Runtime Metrics 277 Introducing Prometheus for Java Developers 278 Prometheus Architecture Overview 278 Using Prometheus with Micrometer 280 Introducing OpenTelemetry 285 What Is OpenTelemetry? 286 Why Choose OTel? 288 OTLP 289 x | Table of Contents

📄 Page 13

The Collector 290 OpenTelemetry Tracing in Java 292 Manual Tracing 292 Automatic Tracing 296 Sampling Traces 297 OpenTelemetry Metrics in Java 298 OpenTelemetry Logs in Java 300 Summary 302 12. Profiling. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305 Introduction to Profiling 306 GUI Profiling Tools 308 VisualVM 308 JDK Mission Control 310 Sampling and Safepointing Bias 314 Modern Profilers 316 perf 316 Async Profiler 319 JDK Flight Recorder (JFR) 321 Operational Aspects of Profiling 325 Using JFR As an Operational Tool 325 Red Hat Cryostat 326 JFR and OTel Profiling 327 Choosing a Profiler 328 Memory Profiling 329 Allocation Profiling 329 Heap Dumps 332 Summary 334 13. Concurrent Performance Techniques. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335 Introduction to Parallelism 337 Amdahl’s Law 338 Fundamental Java Concurrency 339 Understanding the JMM 342 Building Concurrency Libraries 347 Method and Var Handles 348 Atomics and CAS 353 Locks and Spinlocks 354 Summary of Concurrent Libraries 355 Locks in java.util.concurrent 356 Read/Write Locks 358 Table of Contents | xi

📄 Page 14

Semaphores 359 Concurrent Collections 359 Latches and Barriers 360 Executors and Task Abstraction 362 Introducing Asynchronous Execution 362 Selecting an ExecutorService 363 Fork/Join and Parallel Streams 364 Actor-Based Techniques 367 Virtual Threads 368 Introduction to Virtual Threads 369 Virtual Thread Concurrency Patterns 373 Summary 375 14. Distributed Systems Techniques and Patterns. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377 Basic Distributed Data Structures 378 Clocks, IDs, and Write-Ahead Logs 378 Two-Phase Commit 379 Object Serialization 381 Data Partitioning and Replication 382 CAP Theorem 383 Consensus Protocols 384 Paxos 384 Raft 386 Distributed Systems Examples 388 Distributed Database—Cassandra 389 In-Memory Data Grid—Infinispan 391 Event Streaming—Kafka 391 Enhancing Fighting Animals 393 Introducing Kafka to Fighting Animals 394 A Simple Hospital Service 397 An Active Hospital 400 Summary 404 15. Modern Performance and The Future. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405 New Concurrency Patterns 405 Structured Concurrency 405 Scoped Values 409 Panama 412 Leyden 414 Images, Constraints, and Condensers 415 Leyden Premain 418 xii | Table of Contents

📄 Page 15

Valhalla 420 Conclusions 425 A. Microbenchmarking. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427 B. Performance Antipatterns Catalog. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443 Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455 Table of Contents | xiii

📄 Page 16

(This page has no text content)

📄 Page 17

Foreword Optimization is a game of “what,” and “why,” and “where,” but mostly “why.” Why is my application slow? Where is the time being spent? Why are so many resources being consumed? What metrics matter? What problem am I actually trying to solve? Why is this innocent-looking microbenchmark misleading? Why do my users need this? Many of us are taught how to program Java, but very few of us are taught how to optimize Java. We may not even know that we need to be able to optimize—until we hit a problem. But optimization is important. There’s a saying that “slow is the new down”; outages are terrible, but sluggish performance is expensive, and annoying, and wasteful, and also pretty terrible. There’s also sustainability to think about; in general, optimized software consumes less energy and requires less hardware to run. The digital world has a substantial carbon footprint, but we can reduce that by optimizing. But people are perhaps the most important reason to optimize. Slow applications frustrate users, and in the worst case, may even drive them—and their business— elsewhere. Studies have found even small deterioration in response times can reduce user satisfaction and retention rates. So how do we make applications go faster? As Ben and James point out in this book, there are no magic fixes or one-click answers. Ben and James do not give a set of recipes for performance tweaks, for the very good reason that this kind of advice goes very stale, very quickly. Instead, Ben and James focus on principles and methodologies. They begin by intro‐ ducing the basic observables of performance: throughput, latency, capacity, utiliza‐ tion, efficiency, scalability, and degradation. It is important to know which of these matter to you, because you cannot optimize them all—or at least, not all at the same time. It’s exceedingly silly to spend time optimizing throughput if your users actually care about latency, or if the main challenge for your business is hardware costs. Performance is about compromise, and about chasing the right rabbit. xv

📄 Page 18

Ben and James give an overview of how to measure performance and how to use statistical analysis to interpret the results. They then dive into a comprehensive over‐ view of how the Java virtual machine works. They cover memory layouts, garbage collection, code execution, and the just-in-time compiler. This is important because performance tuning is a process of discovery and problem-solving, which has to be rooted in an understanding of the whole system. “The whole system” means software, but also hardware, so Ben and James give an excellent primer on modern hardware. In a realistic software deployment, the edge of “the whole” system is not the plastic case of a single computer. Performance analysis must consider the distinctive charac‐ teristics of cloud environments and the risks and rewards of distributed computing. It’s a lot, but Ben and James cover it. This is a broad book because what a modern software engineer needs to know is equally broad. When trying to solve a performance problem, it’s easy to be distracted by shiny things or influenced by our own pre-judgments. Especially in a complex system, one can easily end up chasing the wrong rabbit. To identify the right rabbit, it’s necessary to go wide, across a diffuse and distributed application, and deep, into the internals of Java garbage collection and threading. One must be systematic and rigorous but also, as Ben and James point out, empathetic and business-aware. Performance is not an absolute quantity; slow only matters if it has consequences. In Spice World, Roger Moore’s character advised that “when the rabbit of chaos is pursued by the ferret of disorder through the fields of anarchy, it is time to hang your pants on the hook of darkness.” With the help of this book, I believe you will be able to chase the right rabbit and hang your pants on the hooks of reason and understanding. — Holly Cummins Senior Principal Software Engineer, Red Hat Middleware xvi | Foreword

📄 Page 19

Preface Why Did We Write This Book? This book is an updated edition of our Optimizing Java title, which was released in 2018. The world has changed markedly since then—in many ways. For Java program‐ mers, the cloud has become ever-more important, and it is now probably more likely than not that Java applications are deployed in the cloud. Cloud native deployment fundamentally changes a number of aspects of performance engineering (or whatever we want to call this specialism), so it seemed appropriate to produce a new edition of the book that reorients the material toward this new reality. Why Should You Read This Book? Java developers have, in many cases, not necessarily had a lot to do with the deploy‐ ment and management of their applications in production. That is, they have tended not to be trailblazers in the adoption of trends like DevOps. With the increasing tide of cloud adoption, this has led to a possible knowledge gap, which this book aims to fix. Alternatively, DevOps professionals may not have had much exposure to Java/JVM technologies but are now finding themselves needing to manage Java applications, or pieces of infrastructure that are implemented in it (e.g., Cassandra, Infinispan, Kafka, etc.). Java processes are fundamentally different from those implemented in Go, Python, Node.js, etc. To get the best out of them, you need some level of understanding of those differences and how to work with them. Whichever tradition you have come from, the end goal is the same—to enable you to have confidence in managing your cloud-based production applications and be able to diagnose issues with them as they arise. xvii

📄 Page 20

Who This Book Is For Java performance optimization is of interest to several different groups of professio‐ nals, not just developers. As such, it is important that we provide on-ramps for people who may be coming from different backgrounds and approaching the subject with a different grounding. The sorts of jobs that our readers might do include: • Developers • Application support and operations staff • DevOps engineers • Architects Each of these groups is likely to have a different focus and take on the material, but they all share a common interest in looking after production business applications in the cloud. They will need to understand the performance behavior of both a single-JVM application and a cloud-deployed, distributed system. In this book we will consider cloud deployments to be public cloud, private cloud, and also a mixture of the two. An awareness of performance methodology and the relevant aspects of statistics is also important, so that observability and other performance data can be accurately analyzed once it has been collected. It is also to be expected that the majority of people who read this book will have a need for, or at least an interest in, some of the internals of the systems they support. This understanding is often very important when diagnosing certain types of performance problems, as well as being attractive to the intellectually curious engineer. What You Will Learn The material in this book covers a wide variety of topics. This is because this field extends beyond the boundaries of software development and overlaps into a variety of other fields. What This Book Is Not You will find almost no discussion of the vendor-specific technologies present on the cloud hyperscalers (AWS, Azure, GCP, OpenShift, and so on) in this book. xviii | Preface