Building Event-Driven Microservices Leveraging Organizational Data at Scale (Adam Bellemare) (Z-Library)

Adam Bellemare Foreword by Martin Kleppmann Building Event-Driven Microservices Leveraging Organizational Data at Scale 2nd Edition

ISBN: 979-8-341-62220-3 SOF T WARE ARCHITEC TURE Event-driven microservices offer an optimal approach to harnessing event streams, reacting and responding to changes as they occur across your company. With this fully revised and updated guide, you’ll learn how to apply the principles of event-driven architecture to create event streams and build powerful microservice applications. Author Adam Bellemare takes you through the process of creating event-driven microservice architectures, from first principles all the way to advanced applications. Covering events, event streams, and microservices, this book will give you powerful and reusable patterns for sharing and using important data all across your organization. • The theory and principles of event-driven architectures • How to design and build event-driven microservice architectures to deliver exceptional business value • Event and event-stream design patterns, including schemas and evolution through time • Microservice application patterns, both as singular services and as a collection of multiple services • Tooling and techniques to get your event-driven microservice ecosystem off the ground and set yourself up for success • Integrating event-driven applications into your existing architecture Building Event-Driven Microservices “The definitive guide to data streaming architectures. This book is a must-read for anyone in the field.” Jay Kreps, cofounder and CEO of Confluent “The insider’s secret to managing scale and taming complexity.” K. Scott Morrison, consultant and former CTO of PHEMI Systems “This book will give you everything you need to know to wrap your head around event-driven systems: from the basic concepts of modeling your data as events all the way to the practical concerns of testing and deploying to production.” Martin Kleppmann, associate professor at the University of Cambridge and author of Designing Data-Intensive Applications Adam Bellemare is a principal technologist at Confluent. In addition to Building Event-Driven Microservices, he’s the author of Building an Event-Driven Data Mesh. Previously, he worked in data platform engineering and data streaming at Shopify and Flipp, where he built, developed, and evangelized event-driven architectures and microservices.

Adam Bellemare Foreword by Martin Kleppmann Building Event-Driven Microservices Leveraging Organizational Data at Scale SECOND EDITION

979-8-341-62220-3 [LSI] Building Event-Driven Microservices by Adam Bellemare Copyright © 2025 Adam Bellemare. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 141 Stony Circle, Suite 195, Santa Rosa, CA 95401. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com. Acquisitions Editor: Louise Corrigan Development Editor: Corbin Collins Production Editor: Clare Laylock Copyeditor: Kim Cofer Proofreader: Vanessa Moore Indexer: Potomac Indexing, LLC Cover Designer: Karen Montgomery Cover Illustrator: Karen Montgomery Interior Designer: David Futato Interior Illustrator: O’Reilly Media, Inc. August 2020: First Edition September 2025: Second Edition Revision History for the Second Edition 2025-09-15: First Release See https://www.oreilly.com/catalog/errata.csp?isbn=0642572147761 for release details. The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Building Event-Driven Microservices, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc. The views expressed in this work are those of the author, and do not represent the publisher’s views. While the publisher and the author have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights. This work is part of a collaboration between O’Reilly and Confluent. See our statement of editorial independence.

Table of Contents Foreword. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii Part I. Introduction to Event-Driven Microservices 1. Why Event-Driven Microservices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 What Are Event-Driven Microservices? 3 Introduction to Domain-Driven Design and Bounded Contexts 6 Microservices, Boundaries, and Communication Structures 9 Business Communication Structures 9 Implementation Communication Structures 10 Data Communication Structures 10 Conway’s Law and Communication Structures 11 Communication Structures in Traditional Computing 13 Option 1: Make a New Service 13 Option 2: Add It to the Existing Service 13 Pros and Cons of Each Option 13 The Team Scenario, Continued 15 Event-Driven Data Communication 15 Example Team Using Event-Driven Microservices 17 Request-Response Microservices 18 Drawbacks of Request-Response Microservices 18 Benefits of Request-Response Microservices 20 Summary 20 iii

2. Fundamentals of Events and Event Streams. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 What’s an Event? 21 What’s an Event Stream? 22 Ephemeral Messaging 24 Queuing 25 Queues via Modern Queue Brokers 26 Queues via Apache Kafka 27 The Structure of an Event 29 Unkeyed Events 30 Keyed Events 30 Entity Events 32 Repartitioning Event Streams 33 Copartitioning Event Streams 34 Aggregating State from Keyed Events 35 Materializing State from Entity Events 36 Deleting Events and Event-Stream Compaction 37 The Kappa Architecture 39 The Lambda Architecture 42 Event Data Definitions and Schemas 44 Powering Microservices with the Event Broker 45 Selecting an Event Broker 46 Support Tooling 46 Hosted Services 46 Client Libraries and Processing Frameworks 46 Community Support 47 Indefinite and Tiered Storage 47 Summary 47 3. Fundamentals of Event-Driven Microservices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 The Basics of Event-Driven Microservices 49 The Basic Producer/Consumer 51 The Stream-Processing Event-Driven Microservice 53 The Streaming SQL Query 55 The Legacy Application 57 Topologies and Event-Driven Microservices 58 Microservice Topology 58 Business Topology 58 Event-Driven Microservice Responsibilities 59 How Small Should a Microservice Be? 61 Managing Microservices at Scale 62 Putting Microservices into Containers 63 iv | Table of Contents

Putting Microservices into Virtual Machines 63 Managing Containers and Virtual Machines 64 Paying the Microservice Tax 65 Service Contracts 66 Summary 67 Part II. Events and Event Streams 4. Schemas and Data Contracts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 A Brief Introduction to Serialization and Deserialization 72 What Is a Schema? 73 What Are Your Schema Technology Options? 77 Google’s Protocol Buffers, aka Protobuf 78 Apache Avro 80 JSON Schema 81 Schema Evolution: Changing Your Schemas Through Time 82 What Is a Data Contract? 85 Negotiating a Breaking Schema Change 87 Step 1: Design the New Schema 88 Step 2: Consult Your Existing Consumers and Gain Approval 88 Step 3: Plan the Release, Migration, and Deprecation 89 Step 4: Execute the Release 89 The Role of the Schema Registry 90 Field-Level Encryption 93 Integration with Data Catalogs 94 Summary 95 5. Designing Events. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 Introduction to Event Types 97 State Events and Event-Carried State Transfer 99 Current State Events 100 Before/After State Events 102 Delta Events 104 Delta Events for Event Sourcing 104 The Problems with Delta Events 107 Where Do I Use Delta and State Events? 116 Hybrid Events: State with a Bit of Delta 117 Measurement Events 119 Measurement Events Enable Aggregations 120 Measurement Event Sources May Be Lossy 120 Table of Contents | v

Measurement Events Can Power Time-Sensitive Applications 120 Notification Events 121 Event Design for Data Privacy and the Right to Be Forgotten 123 Summary 124 6. Integrating Event-Driven Architectures with Existing Systems. . . . . . . . . . . . . . . . . . 127 What Is Data Liberation? 128 The Dual Write Antipattern 129 Data Liberation Patterns 130 Data Liberation Frameworks 130 Liberating Data Using Change-Data Capture 131 Snapshotting the Initial Data Set State 132 Benefits of Change-Data Capture Using Data Store Logs 134 Drawbacks of Change-Data Capture Using Data Store Logs 134 Liberating Data with a Polling Query 135 Incremental Updating 136 Benefits of Query-Based Updating 136 Drawbacks of Query-Based Updating 137 Liberating Data Using Transactional Outbox Tables 138 Performance Considerations 141 Isolating Internal Data Models 141 Ensuring Schema Compatibility 142 Capturing Change-Data Using Triggers 146 Making Data Definition Changes to Data Sets Under Capture 149 Compromises for Data Liberation 150 Sinking Event Data to Data Stores 152 The Impacts of Sinking and Sourcing on a Business 153 Summary 155 7. Denormalization and Eventification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 Eventification at the Transactional Outbox 160 Eventification in a Dedicated Service 162 What Should Go in the Event? And What Should Stay Out? 164 Slowly Changing Dimensions 166 Type 1: Overwrite with the New Value 166 Type 2: Append the New Value 167 Summary 168 8. Stateful Event-Driven Microservices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 Event-Stream Partitions and Consumer Assignments 170 Repartitioning Event Streams 171 vi | Table of Contents

Copartitioning Event Streams 172 Assigning Partitions Within a Consumer Group 173 Selecting a State Store for Your Microservice 175 Materializing State to an Internal State Store 176 Using a Changelog Event Stream as State Recovery 177 Materializing Global State 178 Advantages of Internal State Stores 179 Disadvantages of Internal State Stores 180 Scaling and Recovery of Internal State 181 Materializing State to an External State Store 184 Advantages of External State 185 Drawbacks of External State 186 Scaling and Recovery with External State Stores 187 Rebuilding the External State Store 188 Transactions and Effectively Once Processing 190 Example: Effectively Once Processing for an Inventory Accounting Service 191 Effectively Once Processing with Client-Broker Transactions 192 Effectively Once Processing Without Client-Broker Transactions 193 Summary 198 9. Deterministic Stream Processing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 Determinism with Event-Driven Workflows 200 Timestamps 200 Synchronizing Distributed Timestamps 201 Event Scheduling and Deterministic Processing 203 Custom Event Schedulers 204 Processing Based on Event Time, Processing Time, and Ingestion Time 204 Timestamp Extraction by the Consumer 205 Request-Response Calls to External Systems 205 Watermarks 205 Watermarks in Parallel Processing 206 Stream Time 208 Out-of-Order and Late-Arriving Events 210 Late Events with Watermarks and Stream Time 211 Causes and Impacts of Out-of-Order Events 212 Time-Sensitive Functions and Windowing 213 Handling Late Events 216 Reprocessing Versus Processing in Near–Real Time 217 Intermittent Failures and Late Events 218 Producer/Event Broker Connectivity Issues 219 Summary and Further Reading 220 Table of Contents | vii

10. Building Workflows with Microservices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221 The Choreography Pattern 222 Modifying a Choreographed Workflow 223 Monitoring a Choreographed Workflow 224 The Orchestration Pattern 226 A Simple Event-Driven Orchestration Example 227 A Simple Request-Response Orchestration Example 229 Modifying an Orchestration Workflow 230 Monitoring an Orchestration Workflow 230 Comparing Request-Response and Event-Driven Orchestration 230 Distributed Transactions and the Saga Pattern 231 Sagas via Choreography 232 Sagas via Orchestration 235 The Compensation Workflow Pattern 237 Implementing Orchestration via a Durable Execution Engine 238 Event Sourcing via the Durable Append-Only Log 240 Further Considerations of Durable Execution 241 Summary 243 Part III. Event-Driven Microservices Frameworks 11. Basic Producer and Consumer Microservices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247 Where Do BPCs Work Well? 248 Integration with Existing Systems Using the Sidecar Pattern 248 Stateful Business Logic That Isn’t Reliant Upon Event Order 249 When the Data Store Does Much of the Work 251 Independent Scaling of the Processing and Data Store 251 Summary 252 12. Heavyweight Framework Microservices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253 A Brief History of Heavyweight Frameworks 254 Example: Session Windowing of Clicks and Views 255 The Inner Workings of Heavyweight Frameworks 260 Benefits and Limitations 262 Cluster Setup Options and Execution Modes 264 Use a Hosted Service 264 Build and Run Your Own Cluster 265 Application Submission Modes 267 Driver Mode 267 Cluster Mode 268 viii | Table of Contents

Handling State and Using Checkpoints 268 Scaling Applications and Handling Event-Stream Partitions 269 Scaling an Application by Restarting It 271 Scaling an Application While It Is Running 272 Autoscaling Applications 274 Recovering from Failures 274 Multitenancy Considerations 274 Languages and Syntax 275 Choosing a Framework 275 Summary 276 13. Lightweight Framework Microservices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277 Example: Joining Products with Brand Data on a Foreign Key 278 Under the Hood of Lightweight Frameworks 280 Event Shuffling and Repartitioning 280 Handling State and Using Changelogs 281 Scaling and Recovering from Failures 282 State Assignment 283 State Replication and Hot Replicas 283 Summary 284 14. Streaming SQL. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285 The Basics of the Continuous Query 286 Turning Streams into Tables for Streaming SQL Queries 286 Example: Enriching Data Using Flink SQL 287 Executing Streaming SQL Code 290 Executing SQL Code from the Command-Line Interface 290 Executing SQL Code from Within the Application 290 Executing SQL Code from a Notebook 291 Executing SQL Code Via a Gateway 293 SQL as a Sidecar 293 User-Defined Function Calls 294 Summary 295 15. Microservices Using Functions as a Service. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297 Why Would I Use Functions as a Service? 297 Building Microservices with Functions 299 Cold Starts and Warm Starts 300 Termination and Shutdown 301 Choosing a FaaS Provider 301 Starting Functions with Triggers 302 Table of Contents | ix

Triggering Based on New Events: The Event-Stream Listener 303 Triggering Based on Consumer Group Lag 304 Triggering on a Schedule 306 Triggering Using Webhooks 306 Triggering on Resource Events 306 Scaling Your FaaS Solutions 306 Maintaining State 307 Functions Calling Other Functions 308 Asynchronous Event-Driven Communication Pattern 308 Direct-Call Pattern 309 Handling FaaS Function Failures 311 Durable Function Orchestrators 314 Encoding the Orchestrator in a Proprietary Document 314 Encoding the Orchestrator Within the Source Code 318 Summary 320 Part IV. Consistency, Bad Data, and Supportive Tooling 16. Eventual Consistency. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323 Converging on Consistency, One Event at a Time 325 Strategies for Dealing with Eventual Consistency 329 Use State Event Types 329 Expose Eventual Consistency in the Server Response 329 Use Event-Driven Communication Instead of Request-Response 331 Prevent Failures to Avoid Inconsistency 331 Summary 332 17. Integrating Event-Driven and Request-Response Microservices. . . . . . . . . . . . . . . . . 333 Turning Requests into Events 334 Turning Requests into Events Using a REST Proxy 336 Integrating with Third-Party Request-Response APIs 338 Serving Data Using a Request-Response API 341 Serving Requests with Internal State Stores 341 Serving Requests with External State Stores 345 Handling Requests Within an Event-Driven Workflow 348 Processing Events for User Interfaces 349 Example: Newspaper Publishing Workflow (Approval Pattern) 351 Example: Separating the Editor and Advertiser Approval Services 354 Micro Frontends for Request-Response Applications 356 Example: Experience Search and Review Application 358 x | Table of Contents

The Benefits of Micro Frontends 361 The Drawbacks of Micro Frontends 362 Summary 362 18. Handling Bad Data in Event Streams. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365 The Main Types of Bad Data in Event Streams 366 Type 1: Corrupted Data 366 Type 2: Event Has No Schema 366 Type 3: Event Has an Invalid Schema 366 Type 4: Incompatible Schema Evolution 367 Type 5: Logically Invalid Value in a Field 368 Type 6: Logically Valid Value but Semantically Incorrect 368 Type 7: Missing Events 369 Type 8: Events That Should Not Have Been Produced 369 Preventing Bad Data with Schemas, Validation, and Tests 370 Preventing Bad Data Types 1–5 with Schemas and Schema Evolution 370 Data Quality Rules: Handling Type 6: (Logically Valid But Semantically Incorrect) 371 Testing: Handling Types 7 (Missing Data) and 8 (Data That Should Not Have Been Produced) 372 The Role of Event Design in Fixing Bad Data 372 Fix It Once and Fix It Right with State Events 374 Build Forward: Undo Bad Deltas with New Deltas 376 The Last Resort: Rewind, Rebuild, and Retry 377 Rewind, Rebuild, and Retry from an External Source 377 Rewind, Rebuild, and Retry with the Topic as the Source 378 Summary 379 19. Supportive Tooling. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381 Choosing Your Infrastructure: Build Versus Buy 382 Infrastructure as Code for Clusters and Services 382 Identity and Access Management 384 Microservice-to-Team Assignment System 385 Event-Stream Creation and Modification 385 Event Stream and Microservice Metadata Tagging 386 Quotas 387 Schema Registry 388 Managing Event-Stream Permissions 388 Schema Creation and Modification Notifications 390 Consumer Offset Management 390 State Management and Application Reset 391 Table of Contents | xi

Consumer Offset Lag Monitoring 392 Streamline the Microservice Creation Process 393 Container Management Controls 393 Multicluster Deployments and Event-Stream Replication 394 Dependency Tracking and Topology Visualization 396 Summary 400 20. Testing Event-Driven Microservices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401 General Testing Principles 401 Unit-Testing Microservice Functions 401 Unit Testing Stateless Functions 402 Unit Testing Stateful Functions 403 Testing the Full Event-Driven Topology 404 Testing Schema Evolution and Compatibility 406 Integration Testing of Event-Driven Microservices 408 Local Integration Testing 409 Create a Testing Environment Within the Runtime of Your Test Code 411 Creating a Testing Environment with Containers 413 Integrating Hosted, Managed, and SaaS Services 415 Testing Using Fully Remote Cloud Services 416 Programmatically Create a Temporary Single-Tenant Testing Environment 417 Testing Using a Durable Multitenant Environment 421 Testing Using the Production Environment 422 Choosing Your Full-Remote Integration Testing Strategy 423 Summary 424 21. Deploying Event-Driven Microservices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425 Principles of Microservice Deployment 425 Architectural Components of Microservice Deployment 426 Continuous Integration, Delivery, and Deployment Systems 427 Container Management Systems and Commodity Hardware 427 The Basic Full-Stop Deployment Pattern 428 The Rolling Update Pattern 430 The Breaking Schema Change Pattern 431 Variant 1: Write the Breaking Events to the Same Old Stream 431 Variant 2: Write the Breaking Events to a New Stream 432 Variant 3: Write Both the Old and New Events in Parallel 433 Reprocessing Historical Data for Breaking Schema Changes 435 The Blue-Green Deployment Pattern 436 Summary 438 xii | Table of Contents

22. Conclusion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439 The Data Communication Structure 439 Business Domains and Bounded Contexts 440 Tooling, Infrastructure, and the Microservice Tax 440 Event Design 441 Schematized Events 442 Data Liberation and the Single Source of Truth 442 Microservices 443 Microservice Implementation Options 444 Testing 444 Deploying 445 Final Words 446 Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 447 Table of Contents | xiii

Foreword I am a big fan of the event-driven microservices architecture. I was introduced to it when I was an engineer at LinkedIn in 2012, around the time when Apache Kafka was brand new and stream processing based on Kafka was just getting started. I gave a conference talk about it called “Turning the Database Inside-Out” in 2014, which is still my most-viewed talk on YouTube. And when I wrote my book Designing Data- Intensive Applications (O’Reilly, 2017) I started seeing event streams everywhere, in all the data systems we use every day. Database durability, replication for high availabil‐ ity, transactions, real-time collaboration, updating large-scale analytics systems … it’s all event logs. In the decade since then, I have moved to academia, and I am now on the faculty of the Department of Computer Science and Technology at the University of Cam‐ bridge. Although I’ve shifted research topics a bit, I’m still working on distributed systems, and I still keep coming back to events. They enable scalability, fault toler‐ ance, decentralization, decoupling, debugability, evolvability, and many other won‐ derful things. Events, events, events! They’re great! Use them! Given my excitement for the topic, I was, of course, delighted when Adam asked me to write a foreword for the second edition of his book. He has been in the data engineering space for 15 years too and started working with Apache Kafka in 2015 after version 0.8.2 came out. He built event-driven data pipelines and microservices platforms in several ecommerce organizations, bridging data from the operational to the analytical space, before ending up as a principal technologist at Confluent. He is a great writer, and he has deep insights into all the practical details that go into building scalable and reliable production systems. However, getting your head around event-driven systems can be challenging because it’s a shift in mindset. Most of us are used to the request-response model of services, since that’s how most database queries, REST APIs, and service calls work: you make a request and get back a response. The request-response architecture is so common that it’s easy to take it for granted. You might think it’s the only way of doing things. xv

The trouble with request-response is: what if the data changes after you got the response? The only way you can find out is by sending another request—in other words, by polling. The more often you poll, the quicker you find out if something changed. But most of the time, nothing changed, which makes frequent polling inefficient. Really, it would be so nice if you could simply be notified when something changed, so that you don’t have to keep asking. And that’s what event-driven systems are all about. This book will give you everything you need to know to wrap your head around event-driven systems: from the basic concepts of modeling your data as events all the way to the practical concerns of testing and deploying to production. You will learn about best practices, schemas, and the trade-offs and benefits of various approaches. You will see a selection of languages and frameworks, including basic producer/con‐ sumer frameworks, streaming frameworks, functions as a service, streaming SQL, change-data capture, and more. Not all services should be event-driven: traditional microservices and monoliths have their place too. But for the situations where event-driven microservices are a good fit, they really shine. Read on to find out what those situations are and what benefits you can get from event-driven microservices: they enable better communication of data between systems managed by different teams; they allow the same data to be interpreted in multiple ways depending on your business use case; they give you scalability and resilience by decoupling services from each other; and they give you the ability to respond to things in real time as they happen. Given how powerful this approach is, I’m so glad that Adam has written this book so that you can learn all of these things too, allowing you to make well-informed decisions on when and how to go event-driven. Hope you enjoy the book! — Martin Kleppmann Associate professor at the University of Cambridge Author of Designing Data-Intensive Applications Cambridge, UK, August 2025 xvi | Foreword

Preface I first wrote this book to be the one that I wish I’d had when I started out on my journey into the world of event-driven microservices. While I wrote much of it based on my own experiences, I was very fortunate to learn and grow alongside many great colleagues and mentors. I’ve done my best to distill everything I know about event-driven microservices into this book, and I hope that it serves you well on your own journey. In the five intervening years between this second edition and the first, I’ve had the opportunity to live and experience an even wider and more detailed world of event- driven architectures and microservices. As a Principal Technologist at Confluent, I’ve been privileged to engage with countless amazing organizations doing some really remarkable things—some as our customers, and some as members of the vibrant Apache Kafka community. While many factors have influenced this second edition, two stand above the rest. The first is the content that I wish I had put in for the first edition, including detailed examinations of schemas, event design, orchestration, and eventual consis‐ tency. Though primarily theoretical in nature, these subjects prove to be essential for building well-designed event-driven microservices. The second major influencing factor is that of dominant technology trends. It is far easier today than it has ever been to get started building event-driven microservices. Many things have changed since the first edition of this book, including: The dominance of Apache Kafka Apache Kafka has solidified itself as the de-facto standard event broker. Notably, there are now many Kafka clones out there that use the Kafka APIs, but have completely custom and proprietary implementations under the hood. These Kafka clones provide deployment flexibility and cost savings at the expense of additional latency. Some clones, like Warpstream, run entirely on cloud storage (AWS S3, GCP, Azure, etc.), forgoing local disk in favor of a cloud native model. xvii