Lean DevOps A Practical Guide to On Demand Service Delivery (Benefield Robert)（Z-Library）
DevOps
Deliver Any Service Far More Effectively, Based on What Customers Really Want As service stacks grow more complex, it becomes even tougher to deliver on-demand IT services that meet customers' expectations for speed, consistency, reliability, security, privacy, and value. Layering on new cloud technologies, architectural approaches, or methodologies can aggravate the problem by widening the gap between what delivery teams think they're delivering and what customers actually experience. In Lean DevOps, technical leader Robert Benefield helps you escape this spiral, reverse bad habits, and regain the situational awareness you need to deliver the right services in the right way. Writing for delivery team members and their leaders, Benefield shows how to improve information flow throughout your organization, so you can move toward your customers' target outcomes. He identifies problems arising from traditional approaches to managing teams, debunks excuses often used to prevent progress, and offers realistic recommendations for everything from requirements to incentives. Understand key dynamics that impact service delivery, and avoid focusing on the wrong issues Give your delivery teams stronger abilities to learn and improve Improve team maturity and implement prerequisites for effective use of automation and AI/ML Optimize key service delivery elements, from instrumentation to queue masters and cycles Organize and manage workflows more effectively Handle governance associated with internal controls and external legal/regulatory requirements Leverage the power of Lean and Mission Command to accelerate innovation, empower subordinates, and drive the outcomes you want
6
Views
Registered users can read the full content for free
Register as a Gaohf Library member to read the complete e-book online for free and enjoy a better reading experience.
Page
1
(This page has no text content)
Page
2
Lean DevOps A01_Benefield_FM_pi-pxx.indd 1 01/06/22 6:22 PM
Page
3
A01_Benefield_FM_pi-pxx.indd 2 01/06/22 6:22 PM This page intentionally left blank
Page
4
Boston • Columbus • New York • San Francisco • Amsterdam • Cape Town Dubai • London • Madrid • Milan • Munich • Paris • Montreal • Toronto • Delhi Mexico City • São Paulo • Sydney • Hong Kong • Seoul • Singapore • Taipei • Tokyo Lean DevOps A Practical Guide to On Demand Service Delivery Robert Benefield A01_Benefield_FM_pi-pxx.indd 3 01/06/22 6:22 PM
Page
5
Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and the publisher was aware of a trademark claim, the designations have been printed with initial capital letters or in all capitals. The author and publisher have taken care in the preparation of this book, but make no expressed or implied warranty of any kind and assume no responsibility for errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of the use of the information or programs contained herein. For information about buying this title in bulk quantities, or for special sales opportuni- ties (which may include electronic versions; custom cover designs; and content particular to your business, training goals, marketing focus, or branding interests), please contact our corporate sales department at corpsales@pearsoned.com or (800) 382-3419. For government sales inquiries, please contact governmentsales@pearsoned.com. For questions about sales outside the U.S., please contact intlcs@pearson.com. Visit us on the Web: informit.com/aw Library of Congress Control Number: 2022937306 Copyright © 2023 Pearson Education, Inc. Cover image: Vinap/Shutterstock All rights reserved. This publication is protected by copyright, and permission must be obtained from the publisher prior to any prohibited reproduction, storage in a retrieval system, or transmission in any form or by any means, electronic, mechanical, photocopy- ing, recording, or likewise. For information regarding permissions, request forms and the appropriate contacts within the Pearson Education Global Rights & Permissions Depart- ment, please visit www.pearson.com/permissions. ISBN-13: 978-0-13-384750-5 ISBN-10: 0-13-384750-0 ScoutAutomatedPrintCode A01_Benefield_FM_pi-pxx.indd 4 01/06/22 6:22 PM
Page
6
Author: To my beautiful love, Gabrielle, my “little people” Aidan and Talia, and all of those who have enriched my life and, in the process, made the world a better place. A01_Benefield_FM_pi-pxx.indd 5 01/06/22 6:22 PM
Page
7
A01_Benefield_FM_pi-pxx.indd 2 01/06/22 6:22 PM This page intentionally left blank
Page
8
• vii • Contents Foreword xv Acknowledgments xviii About the Author xx Introduction 1 Chapter 1: The Problem with IT Service Delivery 7 Approach #1: Reduce Delivery Friction 9 The Downsides of Targeting Delivery Friction 11 Approach #2: Managing Service Delivery Risk 12 The Downsides of Targeting Service Delivery Risk 14 The Essence of Delivery 15 Beginning the DevOps Journey 17 Summary 18 Chapter 2: How We Make Decisions 21 Examining the Decision-Making Process 22 Boyd and the Decision Process 23 The OODA Loop 26 The Ingredients of Decision Making 29 Ingredient 1: The Target Outcome 30 Delivering Measures over Outcomes 36 Ingredient 2: Friction Elimination 39 Ingredient 3: Situational Awareness 42 The Challenge of Trust 44 The Fragility of Mental Models and Cognitive Biases 45 Ingredient 4: Learning 48 Failing to Learn 48 A01_Benefield_FM_pi-pxx.indd 7 01/06/22 6:22 PM
Page
9
C O N T E N T S• viii • The Pathway to Improved Decision Making 53 Summary 54 Chapter 3: Mission Command 55 The Origins of Mission Command 56 Learning How to Lead Effectively the Hard Way 57 Managing Through Unpredictability 58 Knowledge and Awareness Weaknesses 59 Misalignments 60 Misjudgment of Ecosystem Complexity 61 The Anatomy of Mission Command 62 Commander’s Intent 63 Brief 66 Situational Overview 67 Statement of the Desired Outcome or Overall Mission Objective 67 Execution Priorities 67 Anti-Goals and Constraints 68 Backbriefing 69 Einheit: The Power of Mutual Trust 71 Creating Einheit in DevOps 74 Continual Improvement 75 Staff Rides 78 After Action Reviews 79 Organizational Impacts of Mission Command 80 Summary 81 Chapter 4: Friction 83 Understanding Ohno’s Forms of Waste 84 Muda (Pure Waste) 86 Muri (Overburden) 109 Mura (Fluctuation and Irregularity) 113 See the Whole 125 Summary 126 A01_Benefield_FM_pi-pxx.indd 8 01/06/22 6:22 PM
Page
10
C O N T E N T S • ix • Chapter 5: Risk 127 Cynefin and Decision Making 128 Ordered Systems 131 Unordered Systems 134 Reimagining Risk Management 143 Have Clear and Understood Target Outcomes 144 Make the Best Choice the Easiest Choice 145 Continually Improve Ecosystem Observability 147 Summary 151 Chapter 6: Situational Awareness 153 Making Sense of Our Ecosystem 154 The Mental Model 157 The Problems with Mental Models 158 Cognitive Bias 161 Gaining Better Situational Awareness 163 Framing 164 Finding and Fixing Framing Problems 165 Information Flow 169 Why Ecosystem Dynamics Matter 169 Meeting Your Information Flow Needs 172 Analysis and Improvement 181 Summary 182 Chapter 7: Learning 183 The Emergence of Skills Attainment Learning 184 The Rise of the One Right Way 186 Outcome-Directed Learning 188 Creating a Learning Culture 191 Day-to-Day Kata 191 Improvement and Problem-Solving Kata 192 The Coaching Practice 193 Summary 195 A01_Benefield_FM_pi-pxx.indd 9 01/06/22 6:22 PM
Page
11
C O N T E N T S• x • Chapter 8: Embarking on the DevOps Journey 197 The Service Delivery Challenge 204 Traditional Delivery Fog in the Service World 205 The Challenge of the “ilities” 207 The Path to Eliminating Service Delivery Fog 209 The Role of Managers in Eliminating Service Delivery Fog 210 Identifying What You Can or Cannot Know 214 Ways the Team Can Eliminate Service Delivery Fog 219 Summary 220 Chapter 9: Service Delivery Maturity and the Service Engineering Lead 221 Modeling Service Delivery Maturity 223 The Example of Measuring Code Quality 224 Service Delivery Maturity Model Levels 225 Service Delivery Maturity Areas of Interest 228 Configuration Management and Delivery Hygiene 232 Supportability 235 Single Point of Failure Mitigation and Coupling Management 239 Engagement 241 The Service Engineering Lead 243 Why Have a Separate Rotating Role? 244 How the SE Lead Improves Awareness 246 Organizational Configurations with the SE Lead 248 Challenges to Watch Out For 250 Incentivizing Collaboration and Improvement 251 Developers Running Production Services 253 Overcoming the Operational Experience Gap 254 Summary 256 A01_Benefield_FM_pi-pxx.indd 10 01/06/22 6:22 PM
Page
12
C O N T E N T S • xi • Chapter 10: Automation 257 Tooling and Ecosystem Conditions 258 Building Sustainable Conditions 260 5S 261 Seeing Automation 5S in Action 278 Tools & Automation Engineering 283 Organizational Details 285 Workflow and Sync Points 285 Summary 287 Chapter 11: Instrumentation and Observability 289 Determining the “Right” Data 291 Know the Purpose and Value 293 Know the Audience 297 Know the Source 302 Making the Ecosystem Observable 307 Instrumenting for Observability 310 Instrumenting Development 310 Instrumenting Packaging and Dependencies 314 Instrumenting Tooling 316 Instrumenting Environment Change and Configuration Management 317 Instrumenting Testing 319 Instrumenting Production 320 Queryable/Reportable Live Code and Services 321 Presenting Task, Change, Incident, and Problem Records Together 321 Environment Configuration 322 Logging 323 Monitoring 324 Security Tracking and Analysis 325 Service Data 326 A01_Benefield_FM_pi-pxx.indd 11 01/06/22 6:22 PM
Page
13
C O N T E N T S• xii • Pulling It All Together 327 Instrumenting a Wastewater Ecosystem 328 Instrumenting an IT Ecosystem 331 Summary 333 Chapter 12: Workflow 335 Workflow and Situational Awareness 336 Managing Work Through Process 337 Managing Work Organically 339 The Tyranny of Dark Matter 340 Learning to See the Disconnects in Action 343 Resolving Disconnects by Building Context 347 Visualizing the Flow 349 Workflow Board Basics 351 State Columns 352 State Columns for Operations 353 Swim Lanes 355 Task Cards 358 Preventing Dark Matter 359 Using the Board 362 Seeing the Problems 363 Limiting Work in Progress 365 The Limits of a Workflow Board 367 Managing the Board 367 Managing Flow and Improvement 368 Summary 368 Chapter 13: Queue Master 371 An Introduction to the Queue Master 372 Role Mechanics 374 “Follow the Sun” Queue Mastering 384 Queue Master Rollout Challenges 389 A01_Benefield_FM_pi-pxx.indd 12 01/06/22 6:22 PM
Page
14
C O N T E N T S • xiii • Team Members Don’t See the Value 389 More Traditionally Minded Managers Thwarting Rollout 390 Pushy Queue Masters 391 Junior Team Members as Queue Masters 391 Queue Masters Who Struggle to Lead Sync Points 394 Summary 394 Chapter 14: Cycles and Sync Points 395 Inform, Align, Reflect, and Improve 396 Top-Down Alignment Control Approach 397 Alignment Through Iterative Approaches 397 Service Operations Synchronization and Improvement 400 The Tactical Cycle 400 Important Differences Between Kickoffs and Sprint Planning 404 Daily Standup 408 Retrospective 411 General Meeting Structure 413 The Learning and Improvement Discussion 415 The Strategic Cycle 421 Strategic Review 424 General Review Structure 426 A3 Problem Solving for the Strategic Review 427 Summary 432 Chapter 15: Governance 433 Factors for Successful Governance 434 Meeting Intent 435 No Target Outcome Interference 437 Maintain Situational Awareness and Learning 438 Common Governance Mistakes 440 Poor Requirement Drafting and Understanding 440 A01_Benefield_FM_pi-pxx.indd 13 01/06/22 6:22 PM
Page
15
C O N T E N T S• xiv • Using Off-the-Shelf Governance Frameworks 445 Out-of-the-Box Process Tooling and Workflows 450 Tips for Effective DevOps Governance 453 Understand Governance Intent 454 Make It Visible 454 Propose Reasonable Solutions 456 Automation and Compliance 458 Be Flexible and Always Ready to Improve 458 Summary 460 Appendix 461 Index 463 A01_Benefield_FM_pi-pxx.indd 14 01/06/22 6:22 PM
Page
16
• xv • Foreword Just about everyone can remember being inconvenienced by a computer outage at sometime during their lives. I particularly remember walking into the Melbourne airport on a sunny Saturday morning and discovering that the computer which ran all the check-in terminals was down. It stayed down all morning, exposing just how time-consuming and awkward the manual check-in process was. Some of us made it to our destinations many hours late; others were not so lucky. And I remember the hours-long check-in lines at the Baltimore airport many years later; the computer running Southwest’s check-in kiosks was out and this time there was no manual backup process. After multiple long lines, we raced to our gate just in time to board before the door closed on a half-empty plane. As I stood in those lines, I imagined the scene in the data center where “emergency responders” were no doubt scrambling to bring the system back up; after all, I had been there. Some years ago, I managed a factory data center, and we knew that any outage longer than a half hour would seriously curtail pack-out. It only happened once, and believe me, that was one time too many. At last, we have a book written for the people on the emergency response side of a computer outage. It’s not a book focused on being agile or lean; it’s a book that focuses on being ready: ready to prevent serious service delivery problems, ready to limit the damage when they happen, and ready to uncover the cause of any incident and keep it from happening again. You would think, given our increasing reliance on technology, that there would be a lot of information about how to do these things well. But I haven’t seen a lot of guidance for those who hold this challenging job, at least until now. We have decades, if not centuries, of experience handling emergency situations in fire departments, hospitals, and military organizations. But rarely have we thought to apply that experience to the technical world, prob- ably because we don’t consider a technical failure to be a matter of life or death. But as Robert demonstrates throughout this book, lessons from other domains can be surprisingly relevant for those seeking to prevent computer A01_Benefield_FM_pi-pxx.indd 15 01/06/22 6:22 PM
Page
17
F O R E W O R D• xvi • outages and, when they do (inevitably) occur, limit their damage and ensure a rapid, safe recovery. Service delivery teams do a lot more than minimize outages and smoothly handle any that may arise; they provide a critical interface between their organization and its consumers. This book is based on a simple prem- ise—the purpose of a technical system is to bring about expected outcomes for its consumers, and the purpose of service delivery teams is to ensure those expectations are met. They do this by understanding consumers’ intent when using the service, discovering what prevents consumers from achieving the outcomes they expect, and learning how to improve the sys- tem and close the gaps. This book is not about meeting service delivery targets; it’s about pay- ing attention: paying attention to consumers, being aware of their expecta- tions, and being attentive to their frustrations. It’s about paying attention to the ways in which complex technical systems interact, how information flows through organizations, and how decisions are made and executed. It’s about paying attention to and improving the way work gets done and the outcomes that are delivered. Be warned that focusing on delivering outcomes is a novel concept for IT teams, because historically, there was a large time and distance gap between technical teams and their consumers. Most “standard” IT prac- tices, including Agile practices, presume that an intermediary translates consumer intent into proxy features and goals, which then guide the deci- sions of the technical team. This book eliminates such proxies, and in doing so, it may contradict some of your views on appropriate roles, responsibili- ties, and processes. Therefore, you might want to check your confirmation bias at the door as you enter, because, as the saying goes, “It ain’t what you don’t know that gets you into trouble. It’s what you know for sure that just ain’t so.” This is an important book. It is well written and engaging, with great stories and easy-to-understand analogies. It simultaneously challenges read- ers and offers very practical advice. I highly recommend it. Mary Poppendieck January, 2022 A01_Benefield_FM_pi-pxx.indd 16 01/06/22 6:22 PM
Page
18
• xvii • Register your copy of Lean DevOps on the InformIT site for convenient access to updates and/or corrections as they become available. To start the registration process, go to informit.com/register and log in or create an account. Enter the product ISBN (9780133847505) and click Submit. Look on the Registered Products tab for an Access Bonus Content link next to this product, and follow that link to access any available bonus materi- als. If you would like to be notified of exclusive offers on new editions and updates, please check the box to receive email from us. Figure Credits Cover image: Vinap/Shutterstock Figures 2.1, 2.2, 2.4–2.11, 3.1–3.9, 4.1–4.7, 4.9–4.17, 5.1–5.11, 6.1–6.10, 7.1–7.4, 8.1–8.8, 9.2–9.8, 10.1–10.6, 11.1–11.8, 11.10, 12.1–12.9, 13.1–13.8, 14.1–14.7, 15.1–15.4: Courtesy of Gabrielle Benefield Figure 2.3: John R. Boyd A01_Benefield_FM_pi-pxx.indd 17 01/06/22 6:22 PM
Page
19
• xviii • Acknowledgments I am far from a natural at writing a book. Its linear format feels overly con- straining to the way that I think. Books also feel like a permanent record, while life to me feels more like a constant quest to test ideas, gain knowl- edge, and correct mistakes to improve yourself and, if you are lucky, maybe make the world a little bit better along the way. This made creating this book rather arduous. In my desire to help you, the reader, get the most coherent message, I spent an inordinate amount of time rewriting and shuffling content around. I probably would still be at it today if it weren’t for the immense help and patience of several key people along the way. The person most central to helping turn this book into reality, and the one who drew the great illustrations in this book, is my wife and partner- in-crime Gabrielle Benefield. With her product innovation background and my delivery experience, we constantly challenge each other to understand what so often goes wrong with teams and organizations in order to find the best way to help others sustainably deliver the outcomes that really matter. This book, along with the Mobius Loop innovation framework, have come out of years of deep discussion and collaboration to bring out the best in people in order to overcome the problems and dysfunction that so often thwart organizational success. I also cannot thank Mary and Tom Poppendieck enough for their help and inspiration throughout this process. Both so generously spent the time challenging me to dig deep into my experiences to try to best convey to the world why the key pillars of this book are so critical to DevOps success. I want to extend a special thanks to Colonel David A Hopley OBE RM (Retd). I learned and adopted many of the principles of Mission Com- mand by working alongside many talented military veterans early on in my career. Despite my hazy awareness of their origins and understanding of the nuances that make them so effective, the approach not only resonated with A01_Benefield_FM_pi-pxx.indd 18 01/06/22 6:22 PM
Page
20
A C K N O W L E D G M E N T S • xix • me but always seemed to work better than traditionally run teams. Being the formal commanding officer of the UK Special Forces (SBS) and as a member of the teaching staff at the Joint Services’ Defence College, Colonel Hopley helped improve upon my lay description of Mission Command. I also want to thank those who hung in through thick and thin reviewing the many iterations of the manuscript, some who also spent several hours helping me clarify my often-jumbled thoughts. This includes: Steve Free- man, James Duncan Davidson, David McNab, Jeff Sutherland, and Tim Beattie. I am also grateful for the patience of the editorial and production team at Addison-Wesley, including: Haze Humbert, Chris Guzikowski, and Menka Mehta. Your help turned this book into something that I am proud of. Finally, there are a number of people who were instrumental as sound- ing boards to help clarify many of the ideas in this book, including (in no particular order): Peter Webb, Priit Kaasik, Heikko Ellermaa, Martin Sulg, Ainar Sepp, Sarah-Jane Mason, Chris Matts, Ademar Aguiar, and Henrik Kniberg. You helped me express why the approaches in this book not only work, but can help others on their own journey to learn, improve, and maybe make the world a little bit better for all of us. A01_Benefield_FM_pi-pxx.indd 19 01/06/22 6:22 PM
The above is a preview of the first 20 pages. Register to read the complete e-book.
AI Reading Assistant
Whole-book reading guide from stratified index samples; jump to passages in the text
AI guide
【One-Line Pitch】
A practical, systems-minded guide for delivery teams and their leaders who want to close the gap between what IT thinks it ships and what customers actually experience—less about adopting a methodology, more about improving decisions, information flow, and outcomes.
【Book Arc】
- **Opening (~0%–18%)**: Frames the core problem—on-demand IT service delivery is failing not because teams lack tools or methods, but because they misdiagnose the issue. The book opens by examining common but flawed approaches (reducing friction, managing risk) and sets up the DevOps journey as a response.
- **Early (~18%–36%)**: Builds the conceptual foundation: how decisions are made (drawing on Boyd's OODA loop), the ingredients of good decisions (target outcomes, friction elimination, situational awareness, learning), Mission Command as a leadership model, and Lean's forms of waste (muda, muri, mura). This is the theoretical spine of the book.
- **Middle (~36%–55%)**: Moves into practice: service delivery maturity models, the Service Engineering Lead role, automation and 5S, instrumentation and observability, workflow boards, and the Queue Master role. Here the book translates principles into concrete organizational mechanics.
- **Late (~55%–82%)**: Covers cycles and sync points (tactical and strategic), governance (intent, common mistakes, compliance), and closes with a foreword framing the book as being about "being ready" rather than being agile or lean. The emphasis shifts to sustaining improvement and handling real-world constraints.
- **Ending (~82%–100%)**: Acknowledgments and back matter; the excerpts do not cover a formal concluding chapter beyond the governance material.
【Key Takeaways】
- **The real problem is misdiagnosis, not missing tools** (Opening): Teams often target delivery friction or risk in ways that backfire; the book argues for understanding the essence of delivery first—what customers actually experience versus what teams believe they deliver.
- **Decision quality is the lever** (Early): Benefield grounds the book in Boyd's OODA loop and four decision ingredients—target outcomes, friction elimination, situational awareness, and learning. This matters because most delivery dysfunction traces back to poor decisions, not poor effort.
- **Mission Command beats top-down control** (Early): Commander's intent, briefs, backbriefing, anti-goals, and mutual trust (Einheit) give teams autonomy aligned to outcomes—especially useful when ecosystems are unpredictable and knowledge is unevenly distributed.
- **Lean waste has three faces** (Early): Muda (pure waste), muri (overburden), and mura (fluctuation) are treated as distinct problems; seeing the whole system matters more than optimizing one form.
- **Maturity is measurable and role-driven** (Middle): A service delivery maturity model (configuration hygiene, supportability, coupling management, engagement) pairs with a rotating Service Engineering Lead to close the operational experience gap.
- **Observability requires intentional instrumentation** (Middle): The book stresses knowing the purpose, audience, and source of data before instrumenting—across development, packaging, tooling, testing, and production—rather than collecting data indiscriminately.
- **Workflow and queue management make work visible** (Middle): Workflow boards, WIP limits, and the Queue Master role combat "dark matter" (invisible work) and improve flow; the book is candid about rollout challenges like resistant managers and junior queue masters.
- **Governance should serve intent, not obstruct it** (Late): Effective governance understands its purpose, stays visible, avoids off-the-shelf frameworks applied blindly, and remains flexible—automation and compliance can coexist.
【Reading Tips】
- Deep-read the early chapters (decision-making, Mission Command, waste); they carry the book's argument and are referenced throughout. Skim the foreword and acknowledgments.
- Treat the middle chapters as a toolkit: pick the maturity model, instrumentation, or workflow material most relevant to your current pain point rather than reading linearly.
- Watch for the recurring theme that proxies (intermediaries translating customer intent) are a root cause—this challenges standard Agile assumptions and is worth reflecting on.
- The governance chapter is the most immediately actionable for managers; read it alongside the cycles/sync-points material for a complete picture of sustaining improvement.
【Coverage Limits】
The excerpts are heavily front- and back-matter weighted (title pages, copyright, table of contents, foreword, acknowledgments); chapter bodies are represented mainly by their tables of contents, so this guide maps structure and themes rather than detailed arguments or examples within chapters.
Passage locations
Excerpt 1
书名: Lean DevOps A Practical Guide to On Demand Service Delivery (Benefield Robert) (z-library.sk, 1lib.sk, z-lib.sk) 作者: Benefield Robert Deliver Any Service...
View in text
Page 5
n, Inc. Cover image: Vinap/Shutterstock All rights reserved. This publication is protected by copyright, and permission must be obtained from the publisher p...
View in text
Excerpt 3
e Coaching Practice 193 Summary 195 A01_Benefield_FM_pi-pxx.indd 9 01/06/22 6:22 PM C O N T E N T S• x • Chapter 8: Embarking on the DevOps Journey 197 The S...
View in text
Excerpt 4
lenges 389 A01_Benefield_FM_pi-pxx.indd 12 01/06/22 6:22 PM C O N T E N T S • xiii • Team Members Don’t See the Value 389 More Traditionally Minded Managers...
View in text
Recommended for You
{{#thumbnailUrl}}
{{/thumbnailUrl}}
{{^thumbnailUrl}}
{{/thumbnailUrl}}
Loading recommended books...
Failed to load, please try again later
Tip the Site
Scan the WeChat Pay or Alipay code to tip. No login required.
WeChat Pay
Alipay