Semantic Software Design A New Theory and Practical Guide for Modern Architects (Eben Hewitt) (Z-Library)

📄 File Format: PDF

💾 File Size: 8.4 MB

📖 Read Online ⬇️ Download

Registered users can read the full content for free

Register as a Gaohf Library member to read the complete e-book online for free and enjoy a better reading experience.

📄 Page 1

Eben Hewitt Semantic Software Design A New Theory and Practical Guide for Modern Architects

📄 Page 2

(This page has no text content)

📄 Page 3

Eben Hewitt Semantic Software Design A New Theory and Practical Guide for Modern Architects Boston Farnham Sebastopol TokyoBeijing

📄 Page 4

978-1-492-04595-3 [LSI] Semantic Software Design by Eben Hewitt Copyright © 2020 Eben Hewitt. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com. Acquisitions Editors: Ryan Shaw and Chris Guzikowski Development Editor: Alicia Young Production Editor: Kristen Brown Copyeditor: Octal Publishing, LLC Proofreader: Charles Roumeliotis Indexer: Ellen Troutman-Zaig Interior Designer: David Futato Cover Designer: Karen Montgomery Illustrator: Rebecca Demarest October 2019: First Edition Revision History for the First Edition 2019-09-25: First Release See http://oreilly.com/catalog/errata.csp?isbn=9781492045953 for release details. The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Semantic Software Design, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc. The views expressed in this work are those of the author, and do not represent the publisher’s views. While the publisher and the author have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights.

📄 Page 5

Table of Contents Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix Part I. Episteme: The Philosophy of Design 1. Origins of Software Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Software’s Conceptual Origins 3 Copies and Creativity 9 Why Software Projects Fail 10 The Impact of Failures 13 2. The Production of Concepts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 Semantics and the Software Factory 17 The Myth of Requirements 19 Semantics and Software Architecture 20 The Semantic Field 21 Designers Are Producers of Concepts 23 Designing Concepts 24 What Is a Concept? 25 Accomplish, Avoid, Fix 26 Outlining Your Concept on the Concept Canvas 26 Ideas Are Captured in a Lookbook 30 Fit to Purpose 32 The Concept Is Expressed in a Parti 33 An Example 35 Adding Aspects to the Parti 36 The Parti Is Based on a Series of Reveals 36 Understanding Ideas 38 iii

📄 Page 6

Sense Certainty 38 Metacognition 39 Context 41 Sets 43 Relations 44 Advantages of Semantic Design 45 3. Deconstruction and Design. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 Introduction to Deconstruction 49 Simplexity 53 (De)composition 55 Affordance 57 Give Intention and Use Value to Negative Space 58 Give Design Decisions at Least Two Justifications 61 Design from Multiple Perspectives 62 Create a Quarantine or Embassy 63 Design for Failure 63 Design Language 64 Naming 64 Start Opposite the User 65 Platforms 66 Disappearing 66 Part II. Semantic Design in Practice 4. Design Thinking. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 Why Design Thinking? 71 Exploring Design Thinking 72 Principles 73 The Method 74 Implementing the Method 81 Summary 84 5. Semantic Design Practices and Artifacts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 Design Principles 86 Pair Designing 88 Murals 89 Vision Box 93 Mind Maps 94 Use Cases 95 Guidelines and Conventions 96 iv | Table of Contents

📄 Page 7

Utils 98 Domain 98 service-api 98 service-impl 99 service-client 99 Approaches 99 Design Definition Document 100 Considerations for Composing Your Design Definition 108 Position Papers 111 RAID 112 Presentations and Multiple Viewpoints 113 Summary 115 6. The Business Aspect. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 Capturing the Business Strategy 120 Provide a Common Understanding 120 Align Strategic Objectives and Tactical Demands 122 Framework Introduction 123 Scope of the Framework 124 Create the Business Glossary 125 Create the Organizational Map 125 Create a Business Capabilities Model 126 Create a Process Map 129 Reengineer Processes 129 Take Inventory of Systems 131 Define the Metrics 131 Institute Appropriate Governance 132 Business Architecture in Applications 133 Summary 136 7. The Application Aspect. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 Embrace Constraints 140 Decouple User Interfaces 141 UI Packages 142 On Platform Design 142 Service Resources and Representations 144 Domain Language 146 API Guidelines 147 Deconstructed Versioning 148 Cacheability and Idempotence 149 Independently Buildable 151 Strategies and Configurable Services 151 Table of Contents | v

📄 Page 8

Application-Specific Services 153 Communicate Through Services 154 Expect Externalization 154 Design for Resilience 155 Interactive Documentation 157 Anatomy of a Service 158 UI Packages 158 Orchestrations 158 Engines 161 Data Accessors 165 Eventing 165 Structure of an Event Message 168 Contextual Services and Service Mixins 168 Performance Improvement Checklist 170 Separating API from Implementation 171 Languages 172 Radical Immutability 173 Specifications 175 A Comment on Test Automation 178 A Comment on Comments 179 Summary 181 8. The Data Aspect. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 Business Glossary 183 Strategies for Semantic Data Modeling 184 Polyglot Persistence 187 Persistence Scorecard 188 Multimodeling 189 Data Models for Streams 191 Feature Engineering for Machine Learning 193 Classpath Deployment and Network Proxies 195 Peer-to-Peer Persistent Stores 196 Graph Databases 198 OrientDB and Gremlin 199 Data Pipelines 200 Machine Learning Data Pipelines 203 Metadata and Service Metrics 206 Auditing 207 ADA Compliance 207 Summary 208 vi | Table of Contents

📄 Page 9

9. The Infrastructure Aspect. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209 Considerations for Architects 209 DevOps 211 Infrastructure as Code 212 Metrics First 215 Compliance Map 217 Automated Pipelines Also First 217 The Production Multiverse and Feature Toggling 218 Implementing Feature Toggles 219 Multi-Armed Bandits: Machine Learning and Infinite Toggles 221 Infrastructure Design and Documentation Checklist 222 Chaos 224 Stakeholder Diversity and Inside/Out 226 Summary 227 Part III. Operations, Process, and Management 10. The Creative Director. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231 The Semantic Designer’s Role 231 Creative Directors Across Industries 234 In Fashion 235 In Film 236 In Video Games 238 In Advertising 238 In Theater 238 In Technology 239 What’s In a Name? 241 11. Management, Governance, Operations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245 Strategy and Tooling 245 Oblique Strategies 247 Lateral Thinking and Working with Concepts 248 Conceptual Tests 252 Code Reviews 254 Demos 255 The Operational Scorecard 255 The Service-Oriented Organization 258 Cross-Functional Teams 262 The Designed Scalable Business Machine 263 Managing Modernization as a Program 266 Change Management 267 Table of Contents | vii

📄 Page 10

Governance 270 Goals 270 Metrics 270 Service Portfolio 271 Service Inventory and Metadata 271 Service Design Checklist 273 Service Design 273 Service Operations 274 Business Processes 275 Data 275 Errors 276 Performance 276 Security 276 Quality Assurance 277 Availability and Support 277 Deployment 278 Documentation 278 Further Reading on Organizational Design 279 12. The Semantic Design Manifesto. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281 The Manifesto 281 The Four Ideals 285 The Key Practices 286 Opening 292 A. The Semantic Design Toolbox. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293 B. Further Reading. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297 Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303 viii | Table of Contents

📄 Page 11

Preface Thank you kindly for picking up Semantic Software Design. Welcome. This book introduces a new method of software design. It proposes a new way of thinking about how we construct our software. It is primarily focused on large projects, with particular benefit for greenfield software projects or large-scale legacy modernization projects. A software project is said to fail if it does not meet its budget or timeline or deliver the features promised in a usable way. It is incontrovertible, and well documented, that software projects fail at alarming rates. Over the past 20 years, this situation has grown worse, not better. We must do something different to make our software designs more successful. But what? My assumption here is that you’re making business application software and services to be sold as products for customers or you’re working at an in-house IT department. This book is not about missile guidance systems or telephony or firmware. It’s not interested in debates about object-oriented versus functional programming, though it could apply for either realm. It’s certainly not interested in some popular framework or another. And for the sake of clarity, my use of “semantic” here traces back to my philosophical training, and as such, it concerns the matter of signs. “Semantic” here refers more to semiology. It is not related or confined to some notion of Tim Berners- Lee’s concept of the Semantic Web, honorable as that work is. The primary audience is CTOs, CIOs, vice presidents of engineering, architects of all stripes (whether enterprise, application, solution, or otherwise), software develop‐ ment managers, and senior developers who want to become architects. Anyone in technology, including testers, analysts, and executives, can benefit from this book. But there is precious little code in the book. It is written to be understood, and hope‐ fully embraced, by managers, leaders, intellectually curious executives, and anyone working on software projects. That is not quite to say that it’s easy. ix

📄 Page 12

The ideas in this book might appear shocking at times. They are likely to irritate some and perhaps even infuriate others. The ideas will appear as novel, perhaps even for‐ eign and strange in some cases; the ideas will surface as borrowed and recast in other cases, such as in the introduction to Design Thinking. Taken in sum, it’s my bespoke method, cobbled together over many years from a wide array of disparate sources. Most of these ideas stem from my studies in philosophy in graduate school. This book represents a tested version of the ideas, processes, practices, templates, and practical methods that together I call “semantic design.” This approach to software design is proven and it works. Over the past 20 years, I have been privileged to work as CTO, CIO, chief architect, and so on at large, global, public companies and have designed and led the creation of a number of large, mission-critical software projects, winning multiple awards for innovation, and, more important, creating successful software. The ideas presented here in a sense form a catalog of how I approach and perform software design. I’ve employed this approach for well more than a decade, leading the design of software projects for $1 million, $10 million, $35 million, and $50 million. Although this might seem a radical depar‐ ture from traditional ways of thinking about software design, it’s not conjecture or theory: again, it’s proven and it works. It is not, however, obvious. We are forced to use the language we inherit. We know our own name only because someone else told us that’s what it was. For reasons that will become clear, in this book I sometimes use the terms “architect” or “architecture” under erasure, meaning it will appear with a strike, like this: architect. That means that I am forced to use the word for clarity or historical purposes to be communicative, but that it is not presen‐ ted as the intended meaning in the current context. The first part of the book presents a philosophical framing of the method. We high‐ light what problem we’re solving and why. This part is conceptual and provides the theoretical ground. The second part of the book is ruthlessly pragmatic. It offers an array of document templates and repeatable practices that you can use out of the box to employ the ele‐ ments of this method in your own daily work. The third part provides an overview of some ways you manage and govern your soft‐ ware portfolio to help contain the general entropy. The book ends with a manifesto that summarizes concisely the set of principles and practices that comprise this method. Taken altogether, the book represents a combined theoretical frame and a gesture toward its practice. It is not closed, however, and is intended to be taken up as a start‐ ing point, elaborated, and improved upon. This book was written very much as a labor of love. I truly hope you enjoy it and find it useful as you apply the method in your own work. Moreover, I invite you to x | Preface

📄 Page 13

contribute to and advance these ideas. I’d be honored to hear from you at eben@ale‐ theastudio.com or AletheaStudio.com. Conventions Used in This Book The following typographical conventions are used in this book: Italic Indicates new terms, URLs, email addresses, filenames, and file extensions. Constant width Used for program listings, as well as within paragraphs to refer to program ele‐ ments such as variable or function names, databases, data types, environment variables, statements, and keywords. Constant width bold Shows commands or other text that should be typed literally by the user. Constant width italic Shows text that should be replaced with user-supplied values or by values deter‐ mined by context. This element signifies a tip or suggestion. This element signifies a general note. This element indicates a warning or caution. Using Code Examples Supplemental material (code examples, exercises, etc.) is available for download at https://aletheastudio.com. This book is here to help you get your job done. In general, if example code is offered with this book, you may use it in your programs and documentation. You do not Preface | xi

📄 Page 14

need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing a CD-ROM of examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a signifi‐ cant amount of example code from this book into your product’s documentation does require permission. We appreciate, but do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: “Semantic Software Design by Eben Hewitt (O’Reilly). Copyright 2020 Eben Hewitt, 978-1-492-04595-3.” If you feel your use of code examples falls outside fair use or the permission given above, feel free to contact us at permissions@oreilly.com. O’Reilly Online Learning For almost 40 years, O’Reilly Media has provided technology and business training, knowledge, and insight to help compa‐ nies succeed. Our unique network of experts and innovators share their knowledge and expertise through books, articles, conferences, and our online learning platform. O’Reilly’s online learning platform gives you on-demand access to live training courses, in- depth learning paths, interactive coding environments, and a vast collection of text and video from O’Reilly and 200+ other publishers. For more information, please visit http://oreilly.com. How to Contact Us Please address comments and questions concerning this book to the publisher: O’Reilly Media, Inc. 1005 Gravenstein Highway North Sebastopol, CA 95472 800-998-9938 (in the United States or Canada) 707-829-0515 (international or local) 707-829-0104 (fax) We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at https://oreil.ly/semantic-software-design. xii | Preface

📄 Page 15

To comment or ask technical questions about this book, send email to bookquestions@oreilly.com. For more information about our books, courses, conferences, and news, see our web‐ site at http://www.oreilly.com. Find us on Facebook: http://facebook.com/oreilly Follow us on Twitter: http://twitter.com/oreillymedia Watch us on YouTube: http://www.youtube.com/oreillymedia Acknowledgments Thank you to the gloriously perspicacious Mike Loukides, whose guidance and encouragement has helped to shape these ideas and bring this work to fruition. I am very grateful to know you and work with you. Thank you for all that you do to advance the discourse in our field. Thank you to the incredibly diligent, detail-oriented, assiduous Alicia Young, my development editor at O’Reilly. Your partnership throughout the creation of this book has been terrific; you’ve done so much to improve and focus it. It’s a pleasure to work with you. Thank you to Mary Treseler, Neal Ford, Chris Guzikowski, and the entire Software Architecture Conference team at O’Reilly. These venues you have created make the space and atmosphere where these ideas can be further explored and challenged. Thank you to Tim O’Reilly, for the awesome wonder that is O’Reilly Media. Thank you to our outstanding enterprise architecture team at Sabre. Andrea Baylor, Andy Zecha, Holt Hopkins, Jerry Rossi, Tom Murray and Tom Winrow, I am grateful to work with each of you and for the joy of all of the beautiful, rigorous systems we make together. Thank you to Jonathan Haynes for your reviews of early drafts and your brave comments that helped improve this work. Thanks goes to Clinton Ander‐ son and Justin Ricketts for all of your support. Thank you to my parents, for inspiring in me the joy and practice of writing. Thank you to my teachers, in particular Christine Ney and Bryan Short. I cherish you for caring enough about the world of ideas to push your students so hard. Thank you to Alison Brown for the many important ideas you contributed here and for your amazing nurturing and support of this work. This is for you, as if to say else‐ wise would make it unso. Preface | xiii

📄 Page 16

(This page has no text content)

📄 Page 17

PART I Episteme: The Philosophy of Design In everything, there is a share of everything. —Anaxagoras In this part, we explore the figure of design itself. We examine in new light how our work designing software came to be shaped, and challenge some received views in our industry. We reimagine architecture as the work of creating concepts, and see how to express those concepts working with teams to create effective software designs.

📄 Page 18

(This page has no text content)

📄 Page 19

CHAPTER 1 Origins of Software Architecture We are most of us governed by epistemologies that we know to be wrong. —Gregory Bateson The purpose of this book is to help you design systems well and to help you realize your designs in practice. This book is quite practical and intended to help you do your work better. We must begin theoretically and historically. This chapter is meant to introduce you to a new way of thinking about your role as a software architect that will inform both the rest of this text and the way in which you approach your projects moving forward. Software’s Conceptual Origins We shape our buildings, and thereafter they shape us. —Winston Churchill FADE IN: INT. A CONFERENCE HALL IN GARMISCH GERMANY, OCTOBER 1968 — DAY The scene: The NATO Software Engineering Conference. Fifty international computer professors and crafts- people assembled to determine the state of the industry in software. The use of the phrase soft- ware engineering in the conference name was delib- erately chosen to be “provocative” because at the time the makers of software were considered so far from performing a scientific effort that calling 3

📄 Page 20

themselves “engineers” would be bound to upset the established apple cart. MCILROY We undoubtedly get the short end of the stick in confrontations with hardware people because they are the industrialists and we are the crofters. (pause) The creation of software is backwards as an industry. KOLENCE Agreed. Programming management will continue to deserve its current poor reputation for cost and schedule effectiveness until such time as a more complete understanding of the program design process is achieved. Though these words were spoken, and recorded in the conference minutes in 1968, they would scarce be thought out of place if stated today. At this conference, the idea took hold was that we must make software in an indus‐ trial process. That seemed natural enough, because one of their chief concerns was that software was having trouble defining itself as a field as it pulled away from hardware. At the time, the most incendiary, most scary topic at the conference was “the highly contro‐ versial question of whether software should be priced separately from hardware.” This topic comprised a full day of the four-day conference. This is a way of saying that software didn’t even know it existed as its own field, sepa‐ rate from hardware, a mere 50 years ago. Very smart, accomplished professionals in the field were not sure whether software was even a “thing,” something that had any independent value. Let that sink in for a moment. Software was born from the mother of hardware. For decades, the two were (literally) fused together and could hardly be conceived of as separate matters. One reason is that software at the time was “treated as though it were of no financial value” because it was merely a necessity for the hardware, the true object of desire. Yet today you can buy a desktop computer for $100 that’s more powerful than any computer in the world was in 1968. (At the time of the NATO Conference, a 16-bit computer—that’s two bytes—would cost you around $60,000 in today’s dollars.) 4 | Chapter 1: Origins of Software Architecture