Software Architecture The Hard Parts Modern Trade-Off Analyses for Distributed Architectures (Neal Ford, Mark Richards, Pramod Sadalage etc.) (Z-Library)

📄 File Format: PDF

💾 File Size: 15.7 MB

📖 Read Online ⬇️ Download

Registered users can read the full content for free

Register as a Gaohf Library member to read the complete e-book online for free and enjoy a better reading experience.

📄 Page 1

Software Architecture: The Hard Parts Modern Trade-Off Analyses for Distributed Architectures Neal Ford, Mark Richards, Pramod Sadalage & Zhamak Dehghani

📄 Page 2

(This page has no text content)

📄 Page 3

Praise for Software Architecture: The Hard Parts “This book provides the missing manual around building microservices and analyzing the nuances of architectural decisions throughout the whole tech stack. In this book, you get a catalog of architectural decisions you can make when building your distributed system and what are the pros and cons associated with each decision. This book is a must for every architect that is building modern distributed systems.” —Aleksandar Serafimoski, Lead Consultant, Thoughtworks “It’s a must-read for technologists who are passionate about architecture. Great articulation of patterns.” —Vanya Seth, Head Of Tech, Thoughtworks India “Whether you’re an aspiring architect or an experienced one leading a team, no handwaving, this book will guide you through the specifics of how to succeed in your journey to create enterprise applications and microservices.” —Dr. Venkat Subramaniam, Award-winning Author and Founder of Agile Developer, Inc. “Software Architecture: The Hard Parts provides the reader with valuable insight, practices, and real-world examples on pulling apart highly coupled systems and building them back up again. By gaining effective trade-off analysis skills, you will start to make better architecture decisions.” —Joost van Wenen, Managing Partner & Cofounder, Infuze Consulting

📄 Page 4

“I loved reading this comprehensive body of work on distributed architectures! A great mix of solid discussions on fundamental concepts, together with tons of practical advice.” —David Kloet, Independent Software Architect “Splitting a big ball of mud is no easy work. Starting from the code and getting to the data, this book will help you see the services that should be extracted and the services that should remain together.” —Rubén Díaz-Martínez, Software Developer at Codesai “This book will equip you with the theoretical background and with a practical framework to help answer the most difficult questions faced in modern software architecture.” —James Lewis, Technical Director, Thoughtworks

📄 Page 5

Neal Ford, Mark Richards, Pramod Sadalage, and Zhamak Dehghani Software Architecture: The Hard Parts Modern Trade-Off Analysis for Distributed Architectures Boston Farnham Sebastopol TokyoBeijing

📄 Page 6

978-1-492-08689-5 [MBP] Software Architecture: The Hard Parts by Neal Ford, Mark Richards, Pramod Sadalage, and Zhamak Dehghani Copyright © 2022 Neal Ford, Mark Richards, Pramod Sadalage, and Zhamak Dehghani. All rights reserved. Printed in Canada. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com. Acquisitions Editor: Melissa Duffield Development Editor: Nicole Taché Production Editor: Christopher Faucher Copyeditor: Sonia Saruba Proofreader: Sharon Wilkey Indexer: Sue Klefstad Interior Designer: David Futato Cover Designer: Karen Montgomery Illustrator: O’Reilly Media, Inc. October 2021: First Edition Revision History for the First Edition 2021-09-23: First Release See http://oreilly.com/catalog/errata.csp?isbn=9781492086895 for release details. The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Software Architecture: The Hard Parts, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc. The views expressed in this work are those of the authors, and do not represent the publisher’s views. While the publisher and the authors have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the authors disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights.

📄 Page 7

Table of Contents Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi 1. What Happens When There Are No “Best Practices”?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Why “The Hard Parts”? 2 Giving Timeless Advice About Software Architecture 3 The Importance of Data in Architecture 4 Architectural Decision Records 5 Architecture Fitness Functions 6 Using Fitness Functions 7 Architecture Versus Design: Keeping Definitions Simple 13 Introducing the Sysops Squad Saga 15 Nonticketing Workflow 16 Ticketing Workflow 17 A Bad Scenario 17 Sysops Squad Architectural Components 18 Sysops Squad Data Model 19 Part I. Pulling Things Apart 2. Discerning Coupling in Software Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 Architecture (Quantum | Quanta) 28 Independently Deployable 29 High Functional Cohesion 30 High Static Coupling 30 Dynamic Quantum Coupling 38 Sysops Squad Saga: Understanding Quanta 42 v

📄 Page 8

3. Architectural Modularity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 Modularity Drivers 49 Maintainability 50 Testability 54 Deployability 55 Scalability 56 Availability/Fault Tolerance 58 Sysops Squad Saga: Creating a Business Case 59 4. Architectural Decomposition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 Is the Codebase Decomposable? 65 Afferent and Efferent Coupling 66 Abstractness and Instability 67 Distance from the Main Sequence 69 Component-Based Decomposition 71 Tactical Forking 73 Trade-Offs 77 Sysops Squad Saga: Choosing a Decomposition Approach 78 5. Component-Based Decomposition Patterns. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 Identify and Size Components Pattern 84 Pattern Description 84 Fitness Functions for Governance 87 Sysops Squad Saga: Sizing Components 90 Gather Common Domain Components Pattern 94 Pattern Description 94 Fitness Functions for Governance 95 Sysops Squad Saga: Gathering Common Components 97 Flatten Components Pattern 101 Pattern Description 102 Fitness Functions for Governance 107 Sysops Squad Saga: Flattening Components 107 Determine Component Dependencies Pattern 111 Pattern Description 112 Fitness Functions for Governance 117 Sysops Squad Saga: Identifying Component Dependencies 118 Create Component Domains Pattern 120 Pattern Description 121 Fitness Functions for Governance 122 Sysops Squad Saga: Creating Component Domains 123 Create Domain Services Pattern 126 Pattern Description 126 vi | Table of Contents

📄 Page 9

Fitness Functions for Governance 129 Sysops Squad Saga: Creating Domain Services 129 Summary 130 6. Pulling Apart Operational Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 Data Decomposition Drivers 132 Data Disintegrators 133 Data Integrators 146 Sysops Squad Saga: Justifying Database Decomposition 150 Decomposing Monolithic Data 151 Step 1: Analyze Database and Create Data Domains 156 Step 2: Assign Tables to Data Domains 156 Step 3: Separate Database Connections to Data Domains 158 Step 4: Move Schemas to Separate Database Servers 159 Step 5: Switch Over to Independent Database Servers 161 Selecting a Database Type 161 Relational Databases 163 Key-Value Databases 165 Document Databases 167 Column Family Databases 169 Graph Databases 171 NewSQL Databases 173 Cloud Native Databases 175 Time-Series Databases 177 Sysops Squad Saga: Polyglot Databases 179 7. Service Granularity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185 Granularity Disintegrators 188 Service Scope and Function 189 Code Volatility 191 Scalability and Throughput 192 Fault Tolerance 193 Security 195 Extensibility 196 Granularity Integrators 197 Database Transactions 198 Workflow and Choreography 200 Shared Code 203 Data Relationships 205 Finding the Right Balance 208 Sysops Squad Saga: Ticket Assignment Granularity 209 Sysops Squad Saga: Customer Registration Granularity 212 Table of Contents | vii

📄 Page 10

Part II. Putting Things Back Together 8. Reuse Patterns. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219 Code Replication 221 When to Use 223 Shared Library 223 Dependency Management and Change Control 224 Versioning Strategies 225 When To Use 227 Shared Service 228 Change Risk 229 Performance 231 Scalability 232 Fault Tolerance 232 When to Use 234 Sidecars and Service Mesh 234 When to Use 239 Sysops Squad Saga: Common Infrastructure Logic 239 Code Reuse: When Does It Add Value? 242 Reuse via Platforms 244 Sysops Squad Saga: Shared Domain Functionality 244 9. Data Ownership and Distributed Transactions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249 Assigning Data Ownership 250 Single Ownership Scenario 251 Common Ownership Scenario 252 Joint Ownership Scenario 253 Table Split Technique 254 Data Domain Technique 256 Delegate Technique 258 Service Consolidation Technique 261 Data Ownership Summary 262 Distributed Transactions 263 Eventual Consistency Patterns 267 Background Synchronization Pattern 269 Orchestrated Request-Based Pattern 272 Event-Based Pattern 277 Sysops Squad Saga: Data Ownership for Ticket Processing 279 10. Distributed Data Access. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283 Interservice Communication Pattern 285 Column Schema Replication Pattern 287 viii | Table of Contents

📄 Page 11

Replicated Caching Pattern 288 Data Domain Pattern 293 Sysops Squad Saga: Data Access for Ticket Assignment 295 11. Managing Distributed Workflows. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299 Orchestration Communication Style 301 Choreography Communication Style 306 Workflow State Management 311 Trade-Offs Between Orchestration and Choreography 315 State Owner and Coupling 315 Sysops Squad Saga: Managing Workflows 317 12. Transactional Sagas. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323 Transactional Saga Patterns 324 Epic Saga(sao) Pattern 325 Phone Tag Saga(sac) Pattern 330 Fairy Tale Saga(seo) Pattern 333 Time Travel Saga(sec) Pattern 336 Fantasy Fiction Saga(aao) Pattern 340 Horror Story(aac) Pattern 343 Parallel Saga(aeo) Pattern 346 Anthology Saga(aec) Pattern 349 State Management and Eventual Consistency 351 Saga State Machines 352 Techniques for Managing Sagas 356 Sysops Squad Saga: Atomic Transactions and Compensating Updates 358 13. Contracts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365 Strict Versus Loose Contracts 367 Trade-Offs Between Strict and Loose Contracts 370 Contracts in Microservices 372 Stamp Coupling 376 Over-Coupling via Stamp Coupling 376 Bandwidth 377 Stamp Coupling for Workflow Management 378 Sysops Squad Saga: Managing Ticketing Contracts 379 14. Managing Analytical Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381 Previous Approaches 382 The Data Warehouse 382 The Data Lake 386 The Data Mesh 389 Table of Contents | ix

📄 Page 12

Definition of Data Mesh 389 Data Product Quantum 390 Data Mesh, Coupling, and Architecture Quantum 393 When to Use Data Mesh 393 Sysops Squad Saga: Data Mesh 394 15. Build Your Own Trade-Off Analysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399 Finding Entangled Dimensions 401 Coupling 401 Analyze Coupling Points 402 Assess Trade-Offs 403 Trade-Off Techniques 404 Qualitative Versus Quantative Analysis 404 MECE Lists 404 The “Out-of-Context” Trap 405 Model Relevant Domain Cases 408 Prefer Bottom Line over Overwhelming Evidence 410 Avoiding Snake Oil and Evangelism 412 Sysops Squad Saga: Epilogue 416 A. Concept and Term References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417 B. Architecture Decision Record References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419 C. Trade-Off References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 421 Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425 x | Table of Contents

📄 Page 13

Preface When two of your authors, Neal and Mark, were writing the book Fundamentals of Software Architecture, we kept coming across complex examples in architecture that we wanted to cover but that were too difficult. Each one offered no easy solutions but rather a collection of messy trade-offs. We set those examples aside into a pile we called “The Hard Parts.” Once that book was finished, we looked at the now gigantic pile of hard parts and tried to figure out: why are these problems so difficult to solve in modern architectures? We took all the examples and worked through them like architects, applying trade-off analysis for each situation, but also paying attention to the process we used to arrive at the trade-offs. One of our early revelations was the increasing importance of data in architecture decisions: who can/should access data, who can/should write to it, and how to manage the separation of analytical and operational data. To that end, we asked experts in those fields to join us, which allows this book to fully incorporate decision making from both angles: architecture to data and data to architecture. The result is this book: a collection of difficult problems in modern software architec‐ ture, the trade-offs that make the decisions hard, and ultimately an illustrated guide to show you how to apply the same trade-off analysis to your own unique problems. Conventions Used in This Book The following typographical conventions are used in this book: Italic Indicates new terms, URLs, email addresses, filenames, and file paths. Constant width Used for program listings, as well as within paragraphs to refer to program ele‐ ments such as variable or function names, databases, data types, environment variables, statements, and keywords. xi

📄 Page 14

Constant width bold Shows commands or other text that should be typed literally by the user. Constant width italic Shows text that should be replaced with user-supplied values or by values deter‐ mined by context. This element signifies a tip or suggestion. Using Code Examples Supplemental material (code examples, exercises, etc.) is available for download at http://architecturethehardparts.com. If you have a technical question or a problem using the code examples, please send email to bookquestions@oreilly.com. This book is here to help you get your job done. In general, if example code is offered with this book, you may use it in your programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require permission. We appreciate, but generally do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: “Software Architecture: The Hard Parts by Neal Ford, Mark Richards, Pramod Sadalage, and Zhamak Deh‐ ghani (O’Reilly). Copyright 2022 Neal Ford, Mark Richards, Pramod Sadalage, and Zhamak Dehghani, 978-1-492-08689-5.” If you feel your use of code examples falls outside fair use or the permission given above, feel free to contact us at permissions@oreilly.com. xii | Preface

📄 Page 15

O’Reilly Online Learning For more than 40 years, O’Reilly Media has provided technol‐ ogy and business training, knowledge, and insight to help companies succeed. Our unique network of experts and innovators share their knowledge and expertise through books, articles, and our online learning platform. O’Reilly’s online learning platform gives you on-demand access to live training courses, in-depth learning paths, interactive coding environments, and a vast collection of text and video from O’Reilly and 200+ other publishers. For more information, visit http://oreilly.com. How to Contact Us Please address comments and questions concerning this book to the publisher: O’Reilly Media, Inc. 1005 Gravenstein Highway North Sebastopol, CA 95472 800-998-9938 (in the United States or Canada) 707-829-0515 (international or local) 707-829-0104 (fax) We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at https://oreil.ly/sa-the-hard-parts. Email bookquestions@oreilly.com to comment or ask technical questions about this book. For news and information about our books and courses, visit http://oreilly.com. Find us on Facebook: http://facebook.com/oreilly Follow us on Twitter: http://twitter.com/oreillymedia Watch us on YouTube: http://youtube.com/oreillymedia Preface | xiii

📄 Page 16

Acknowledgments Mark and Neal would like to thank all the people who attended our (almost exclu‐ sively online) classes, workshops, conference sessions, and user group meetings, as well as all the other people who listened to versions of this material and provided invaluable feedback. Iterating on new material is especially tough when we can’t do it live, so we appreciate those who commented on the many iterations. We thank the publishing team at O’Reilly, who made this as painless an experience as writing a book can be. We also thank a few random oases of sanity-preserving and idea- sparking groups that have names like Pasty Geeks and the Hacker B&B. Thanks to those who did the technical review of our book—Vanya Seth, Venkat Sub‐ ramanian, Joost van Weenen, Grady Booch, Ruben Diaz, David Kloet, Matt Stein, Danilo Sato, James Lewis, and Sam Newman. Your valuable insights and feedback helped validate our technical content and make this a better book. We especially want to acknowledge the many workers and families impacted by the unexpected global pandemic. As knowledge workers, we faced inconveniences that pale in comparison to the massive disruption and devastation wrought on so many of our friends and colleagues across all walks of life. Our sympathies and appreciation especially go out to health care workers, many of whom never expected to be on the front line of a terrible global tragedy yet handled it nobly. Our collective thanks can never be adequately expressed. Acknowledgments from Mark Richards In addition to the preceding acknowledgments, I once again thank my lovely wife, Rebecca, for putting up with me through yet another book project. Your unending support and advice helped make this book happen, even when it meant taking time away from working on your own novel. You mean the world to me, Rebecca. I also thank my good friend and coauthor Neal Ford. Collaborating with you on the materi‐ als for this book (as well as our last one) was truly a valuable and rewarding experi‐ ence. You are, and always will be, my friend. Acknowledgments from Neal Ford I would like to thank my extended family, Thoughtworks as a collective, and Rebecca Parsons and Martin Fowler as individual parts of it. Thoughtworks is an extraordi‐ nary group of people who manage to produce value for customers while keeping a keen eye toward why things work so that we can improve them. Thoughtworks sup‐ ported this book in many ways and continues to grow Thoughtworkers who chal‐ lenge and inspire me every day. I also thank our neighborhood cocktail club for a regular escape from routine, including the weekly outside, socially distanced versions that helped us all survive the odd time we just lived through. I thank my long-time xiv | Preface

📄 Page 17

friend Norman Zapien, who never ceases to provide enjoyable conversation. Lastly, I thank my wife, Candy, who continues to support this lifestyle that has me staring at things like book writing rather than our cats too much. Acknowledgments from Pramod Sadalage I thank my wife, Rupali, for all the support and understanding, and my lovely girls, Arula and Arhana, for the encouragement; daddy loves you both. All the work I do would not be possible without the clients I work with and various conferences that have helped me iterate on the concepts and content. I thank AvidXchange, the latest client I am working with, for its support and providing great space to iterate on new concepts. I also thank Thoughtworks for its continued support in my life, and Neal Ford, Rebecca Parsons, and Martin Fowler for being amazing mentors; you all make me a better person. Lastly, thank you to my parents, especially my mother, Shobha, whom I miss every day. I miss you, MOM. Acknowledgments from Zhamak Dehghani I thank Mark and Neal for their open invitation to contribute to this amazing body of work. My contribution to this book would not have been possible without the contin‐ uous support of my husband, Adrian, and patience of my daughter, Arianna. I love you both. Preface | xv

📄 Page 18

(This page has no text content)

📄 Page 19

CHAPTER 1 What Happens When There Are No “Best Practices”? Why does a technologist like a software architect present at a conference or write a book? Because they have discovered what is colloquially known as a “best practice,” a term so overused that those who speak it increasingly experience backlash. Regardless of the term, technologists write books when they have figured out a novel solution to a general problem and want to broadcast it to a wider audience. But what happens for that vast set of problems that have no good solutions? Entire classes of problems exist in software architecture that have no general good solutions, but rather present one messy set of trade-offs cast against an (almost) equally messy set. Software developers build outstanding skills in searching online for solutions to a current problem. For example, if they need to figure out how to configure a particular tool in their environment, expert use of Google finds the answer. But that’s not true for architects. For architects, many problems present unique challenges because they conflate the exact environment and circumstances of your organization—what are the chances that someone has encountered exactly this scenario and blogged it or posted it on Stack Overflow? Architects may have wondered why so few books exist about architecture compared to technical topics like frameworks, APIs, and so on. Architects rarely experience common problems but constantly struggle with decision making in novel situations. For architects, every problem is a snowflake. In many cases, the problem is novel not just within a particular organization but rather throughout the world. No books or conference sessions exist for those problems! 1

📄 Page 20

Architects shouldn’t constantly seek out silver-bullet solutions to their problems; they are as rare now as in 1986, when Fred Brooks coined the term: There is no single development, in either technology or management technique, which by itself promises even one order of magnitude [tenfold] improvement within a decade in productivity, in reliability, in simplicity. —Fred Brooks from “No Silver Bullet” Because virtually every problem presents novel challenges, the real job of an architect lies in their ability to objectively determine and assess the set of trade-offs on either side of a consequential decision to resolve it as well as possible. The authors don’t talk about “best solutions” (in this book or in the real world) because “best” implies that an architect has managed to maximize all the possible competing factors within the design. Instead, our tongue-in-cheek advice is as follows: Don’t try to find the best design in software architecture; instead, strive for the least worst combination of trade-offs. Often, the best design an architect can create is the least worst collection of trade-offs —no single architecture characteristics excels as it would alone, but the balance of all the competing architecture characteristics promote project success. Which begs the question: “How can an architect find the least worst combination of trade-offs (and document them effectively)?” This book is primarily about decision making, enabling architects to make better decisions when confronted with novel situations. Why “The Hard Parts”? Why did we call this book Software Architecture: The Hard Parts? Actually, the “hard” in the title performs double duty. First, hard connotes difficult, and architects con‐ stantly face difficult problems that literally (and figuratively) no one has faced before, involving numerous technology decisions with long-term implications layered on top of the interpersonal and political environment where the decision must take place. Second, hard connotes solidity—just as in the separation of hardware and software, the hard one should change much less because it provides the foundation for the soft stuff. Similarly, architects discuss the distinction between architecture and design, where the former is structural and the latter is more easily changed. Thus, in this book, we talk about the foundational parts of architecture. 2 | Chapter 1: What Happens When There Are No “Best Practices”?