The AI Playbook (Eric Siegel) (Z Library)

The AI Playbook

Management on the Cutting Edge Series Abbie Lundberg, series editor Published in cooperation with MIT Sloan Management Review Thomas H. Davenport, The AI Advantage: How to Put the Artificial Intelligence Revolution to Work Gerald C. Kane, Anh Nguyen Phillips, Jonathan Copulsky, and Garth Andrus, The Technology Fallacy: How People Are the Real Key to Digital Transformation Jeanne W. Ross, Cynthia Beath, and Martin Mocker, Designed for Digital: How to Architect Your Business for Sustained Success George S. Day and Paul J. H. Schoemaker, See Sooner, Act Faster: How Vigilant Leaders Thrive in an Era of Digital Turbulence Amit S. Mukherjee, Leading in the Digital World: How to Foster Creativity, Collaboration, and Inclusivity Marco Bertini and Oded Koenigsberg, The Ends Game: How Smart Companies Stop Selling Products and Start Delivering Value Christian Stadler, Julia Hautz, Kurt Matzler, and Stephan Friedrich von den Eichen, Open Strategy: Mastering Disruption from Outside the C- Suite Gerald Kane, Rich Nanda, Anh Nguyen Phillips, and Jonathan Copulsky, The Transformation Myth: Leading Your Organization through Uncertain Times Ron Adner, Winning the Right Game: How to Disrupt, Defend, and Deliver in a Changing World Satish Nambisan and Yadong Luo, The Digital Multinational: Navigating the New Normal in Global Business Ravin Jesuthasan and John W. Boudreau, Work without Jobs: How to Reboot Your Organization’s Work Operating System Mohan Subramaniam, The Future of Competitive Strategy: Unleashing the Power of Data and Digital Ecosystems Chris B. Bingham and Rory M. McDonald, Productive Tensions: How Every Leader Can Tackle Innovation’s Toughest Trade- Offs Thomas H. Davenport and Steven M. Miller, Working with AI: Real Stories of Human- Machine Collaboration Ravi Sarathy, Enterprise Strategy for Blockchain: Lessons in Disruption from Fintech, Supply Chains, and Consumer Industries Lynda Gratton, Redesigning Work: How to Transform Your Organization and Make Hybrid Work for Everyone John Horn, Inside the Competitor’s Mindset: How to Predict Their Next Move and Position Yourself for Success Elizabeth J. Altman, David Kiron, Jeff Schwartz, and Robin Jones, Workforce Ecosystems: Reaching Strategic Goals with People, Partners, and Technologies Barbara H. Wixom, Cynthia M. Beath, and Leslie Owens, Data Is Everybody’s Business: The Fundamentals of Data Monetization Malia C. Lazu, From Intention to Impact: A Practical Guide to Diversity, Equity, and Inclusion Daniel Aronson, The Value of Values: The Hidden Superpower That Drives Business and Career Success Eric Siegel, The AI Playbook: Mastering the Rare Art of Machine Learning Deployment

The AI Playbook Mastering the Rare Art of Machine Learning Deployment Eric Siegel The MIT Press Cambridge, Massachusetts London, England

© 2024 Eric Siegel All rights reserved. No part of this book may be reproduced in any form by any electronic or mechanical means (including photocopying, recording, or information storage and retrieval) without permission in writing from the publisher. The MIT Press would like to thank the anonymous peer reviewers who provided comments on drafts of this book. The generous work of academic experts is essential for establishing the authority and quality of our publications. We acknowledge with gratitude the contributions of these otherwise uncredited readers. This book was set in ITC Stone Serif Std and ITC Stone Sans Std by New Best-set Typesetters Ltd. Library of Congress Cataloging- in- Publication Data Names: Siegel, Eric, 1968– author. Title: The AI playbook : mastering the rare art of machine learning deployment / Eric Siegel. Description: Cambridge, Massachusetts : The MIT Press, [2024] | Series: Management on the cutting edge | Includes bibliographical references and index. Identifiers: LCCN 2023017997 (print) | LCCN 2023017998 (ebook) | ISBN 9780262048903 (hardcover) | ISBN 9780262378130 (epub) | ISBN 9780262378123 (pdf) Subjects: LCSH: Business—Data processing. | Machine learning. Classification: LCC HF5548.2 (ebook) | LCC HF5548.2 .S44865 2024 (print) | DDC 658/.05 23/eng/20230—dc08 LC record available at https://lccn.loc.gov/2023017997 10 9 8 7 6 5 4 3 2 1

This book is dedicated with all my heart to my mother, Lisa Schamberg, and my father, Andrew Siegel.

Contents Series Foreword ix Foreword by Morgan Vawter xi Preface: A Brief History of Why Machine Learning Projects Stall xv Optional FAQ: What This Book Is about and Who It’s For xxi Introduction 1 0 BizML: Six Steps to Machine Learning Deployment 21 1 Value: Establish the Deployment Goal 49 2 Target: Establish the Prediction Goal 63 3 Performance: Establish the Evaluation Metrics 81 4 Fuel: Prepare the Data 113 5 Algorithm: Train the Model 141 6 Launch: Deploy the Model 169 BizML Cheat Sheet 195 Conclusion: ML’s Elevator Pitch, Staff, Timeline, Upkeep, and Ethics 197 Acknowledgments 213 About the Author 217 Index 219 This book’s notes— references, plus resources for further learning— are avail- able at www.bizML.com. For a tutorial glossary that includes the terms introduced within this book and more, see www.MachineLearningGlossary.com.

Series Foreword The world does not lack for management ideas. Thousands of research- ers, practitioners, and other experts produce tens of thousands of arti- cles, books, papers, posts, and podcasts each year. But only a scant few promise to truly move the needle on practice, and fewer still dare to reach into the future of what management will become. It is this rare breed of idea— meaningful to practice, grounded in evidence, and built for the future— that we seek to present in this series. Abbie Lundberg Editor in Chief MIT Sloan Management Review

Foreword There’s almost no business outcome that machine learning cannot help you improve today. From delivering a best- in- class customer and con- sumer experience to fueling productivity, increasing safety, optimizing operations, and improving your employee experience, ML can raise the bar on the metrics that matter across all. Its practical deployment rep- resents the forefront of human progress: improving operations with sci- ence. But where do you start, and how do you ensure what you do start doesn’t end up in the dustbin? Over the course of my career I’ve consulted with over thirty For- tune Global 500 companies on data and analytics, and led global data and analytics organizations at Caterpillar and Unilever. I’ve seen the highs and the lows, including analytics programs that generate tremen- dous value and competitive advantage, and those that never seem to leave the starting gate. In my experience, those companies or teams that struggle to embed analytics at scale typically suffer not because of imperfect analytics execution or ML models, but rather because of a gap in the other factors required for success. As one example, while consulting, I worked with an analytics team at one of the world’s largest retailers on a program to improve market- ing ROI. The in- house team had already developed an advanced media analytics model. They were flush with data, leveraging hundreds of millions of data points on marketing spend, response, products, stores, and other contributing factors. The team poured hours and hours into perfecting the model and fine- tuning it to highest possible levels of

xii Foreword accuracy and then summarizing the output into a list of top insights for action. The day of the big presentation to marketing leadership arrived and the team presented the recommendations to improve ROI by mak- ing key changes to offline marketing spend. They looked to top market- ing leadership for their reaction, expecting smiles, gratitude, praise, and appreciation. Instead, they were met with a mix of apathy and disbe- lief. The problem was that the team had missed crucial steps required to fully understand and incorporate stakeholder priorities, decision- making factors, and processes. Contrast that with an experience I had leading an AI- powered portfo- lio optimization program at Unilever. Unilever is a global organization. The products are sold in over 25 million stores across 190 countries, with over 2.5 billion people using the products every day. Unilever’s brands include Dove, Knorr, Sunsilk, Hellmann’s, Axe, Ben & Jerry’s, Domestos, Suave, TRESemmé, and Magnum. We saw an opportunity to make smarter and faster decisions by tak- ing a global, data- driven approach to optimize our portfolio of prod- ucts and reduce complexity— through a program we would later name Polaris. A sharper portfolio of products ultimately benefits consumers and retailers, optimizes our operations, and drives profitable growth for Unilever’s shareholders. Our team built an AI- powered capability and business process to analyze the entire product portfolio globally and recommend products to delist, grow, fix, and protect. The system leverages analytics to track the execution of those actions and drive accountability across thousands of individuals in the organization. We created and scaled Polaris globally in approximately two years, bringing together the best of machine and human intelligence, which empow- ered us to make more efficient and effective decisions and grow through simplification. The path to get there wasn’t easy and there wasn’t a guidebook available to help us at the time. Fortunately for the reader, the steps outlined in this book bring to life crucial best practices we followed in delivering a globally scaled initiative with lasting business impact. These include:

Foreword xiii 1. Start with outcomes in mind and focus on delivering value incre- mentally. We started with a simple question: Could we increase the rate of decision making and execution to simplify the product portfolio— delivering savings while driving growth with our customers? Only after delivering on that scope and establishing that value did we expand to complete product portfolio optimization, including non- consumer facing simplification such as flagging specifications and ingredients to harmonize across products. 2. Leverage empathy to overcome barriers to change. Consciously or unconsciously, we are all preprogrammed to resist change. To overcome this, the analytics team spent hundreds of hours with other teams across the business to understand how port- folio decisions were being taken currently— including marketing, sales, supply chain, finance, research and development, and retail- ers. By gaining an understanding of the pain points in the current processes, we were able to bring forward a compelling value proposi- tion for stakeholders across levels and functions. 3. Prepare the data so that it meets business needs. Only by anticipating early the differences in data availability, due to the global nature of our business, did the team succeed in scal- ing the capability across geographies. We recognized that we had to adapt to variations of data across markets— some of which were rich with retailer and third- party data illuminating shopper behavior pat- terns, while others held inconsistent point of sale and shopper infor- mation based on the route to market. A versatile data infrastructure and stringent data validation process were key to success. These experiences have made me acutely aware of the many hur- dles that must be overcome to deliver scaled value realization with ML. Innovating the enterprise with ML is revolutionary, and revolutions aren’t easy. Many senior data leaders come to learn the same lessons, but only after years of experience and failed projects. Then after understanding it

xiv Foreword themselves, they still struggle to advocate for these success factors with their business counterparts. Without common understanding between business stakeholders and data leaders on the best practices for deliver- ing data and analytics transformations, many projects fail to take off, struggle to scale, or ultimately don’t deliver on the business outcomes. The industry needs a framework to better leverage ML for business results. This book introduces bizML, which brings forward the best prac- tices in a succinct and actionable way. Not only is the book a timely and much needed addition to the industry; it is also powerful in bringing AI down to earth, eschewing the hype, and making it tangible for all readers. This book is the driver’s manual for machine learning— every business and analytics professional should read it. Morgan Vawter Global Vice President of Data & Analytics at Unilever, former Chief of Analytics at Caterpillar, former Data Management Practice Lead at Accenture, and a Fortune magazine “40 Under 40” honoree

Preface A Brief History of Why Machine Learning Projects Stall When promoting breakthrough technology, be careful what you wish for. Back in the Dark Ages, before data was cool and phones were smart, I networked my way into the swank office of a powerful business execu- tive. Hoping that he would introduce me to— or become— my first cli- ent, I declared that I was striking out on my own as a machine learning (ML) consultant. Unfamiliar with ML and disinterested, he looked at me like, “Don’t waste my time,” and I was quickly back on the streets of San Francisco. This was 2003, right after I’d relocated from the East Coast and ordered new business cards, all in the pursuit of my passion. I had fallen in love with ML a dozen years earlier, first in the research lab and then as a Columbia University professor teaching the graduate- level ML and AI courses. It was the most exciting, potent, and widely applicable kind of technology. Moving west, I vowed to introduce it to the non- academic world. I wanted to see ML deployed. At that time, a corner of the industrial world was already using ML, but they called it something else: data mining. I thought that term was misleading to the non- data folks, but “machine learning” kept getting me kicked out of offices. So I latched onto a new buzzword that had just started to gain traction, predictive analytics. A rose by any other name. Unfortunately, my improved vocabulary didn’t immediately land me clients. “You should just take a full- time job,” a senior executive at an established analytics vendor bluntly threw in my financially insecure face. Instead, I doubled down. Tripled down. I held corporate training seminars. I published articles. I networked like mad.

xvi Preface Clients eventually started coming in, but only enough to keep me busy. I was knee- deep in demand, but I needed it up to my belly button. The world still didn’t get it. I had to evangelize harder. I took a three- pronged approach: 1. Conference. I launched Machine Learning Week (formerly Predictive Analytics World), the first ML conference series outside academic and vendor events. Bolstered by its sister publication, the Machine Learning Times, the conference series has since grown to serve 18,000 attendees internationally. 2. Book. Next, I wrote Predictive Analytics, the first popular book that showed readers of all levels how the algorithms work under the hood. Written to ignite and excite, it ended up becoming a best- seller, winning several awards, landing me 100 keynote speeches at conferences outside my own, and being adopted as course material by hundreds of universities. 3. Music video. I even dropped an educational rap music video called “Predict This!,” which went a bit viral (to watch, go to www.Predict This.org). Surely this proves that I’d literally do anything to spread the gospel of ML. Whether or not these efforts helped light the fuse, one thing’s for sure: ML exploded in popularity. It grew from a nascent industry to a full- blown commercial movement. It came of age as a core enterprise practice necessary to sustain competitive advantage. Hyperboles reigned as data scientist dethroned firefighter to become “the sexiest job.” Watching ML become so hot felt both gratifying and surreal. The experience reinforced an age- old lesson: Keep the faith. When you believe in a good idea— such as the notion that learning from data is not only cool but valuable— and stick to your convictions, people will eventually come around. Failure to Launch Unfortunately, ML’s great rise has also taught me another lesson: Be careful what you wish for. The buzz has gone too far. In a way, ML is now

Preface xvii too hot for its own good. The problem is, the onslaught of excitement has fed a common misconception that derails many ML projects: The ML Fallacy: Since ML algorithms work (amazing and true), the models they generate are intrinsically valuable (not true). The value of ML comes only by launching it to enact organizational change. After generating a model with ML, you capture its potential value only when you deploy it so that it actively improves operations. Until a model is used to actively reshape how your organization works, it’s use- less— literally. A model doesn’t solve any business problems on its own and it ain’t gonna deploy itself. ML can be the disruptive tech- nology it’s cracked up to be, but only if you disrupt with it. Most ML projects fail to deploy. I believe this is mainly because most ML leaders neglect to properly plan for the operational change that deployment would bring to fruition. That planning takes more preaching, socializing, cross- disciplinary collaboration, and change- management panache than many, including myself, initially realized. Far too often, the data scientist delivers a viable model, but the oper- ational team isn’t ready for the pass— and they drop the ball. There are wonderful exceptions and glowing successes, but the generally poor track record we witness today forewarns of broad disillusionment with ML— even a dreaded AI winter. It’s time to tap the brakes and correct course so that ML can deliver on its promise. Breaking through the ML Snafu So I’ve pivoted from ML cheerleader to wary disciplinarian— albeit an optimistic one— with a new mission: Standardize and broadcast the very particular business discipline needed to get ML launched. Whereas my first book was about how ML works technically, this book is about how to run ML projects so that models not only work in the lab but also suc- cessfully deploy. First things first: Business professionals— who are a primary audi- ence for this book— need some edification. Before those in charge can

xviii Preface confidently green- light model deployment, they must gain a concrete understanding of how an ML project works from end to end: What will the model predict? Precisely how will those predictions affect operations? Which metric meaningfully tracks how well it predicts? and What kind of data is needed? Only when the business leaders— including executives, managers, and decision makers— come up to speed on this semi- technical but straightforward knowledge can we bridge the gap between the tech and business sides and bring model deployment into the realm of possibility. These days, everything I do is to unite those two worlds, tech and biz. In addition to this book, I’ve taken another three- pronged approach: 1. Conferences focused on deployment. Newer offshoots of my event series, Machine Learning Week, build on the nuts- and- bolts aspect of ana- lytics to also cover industry- specific deployment, including appli- cations in marketing, financial services, industry 4.0, healthcare, and climate technology. The first track devotes itself to the business side— we call it the operationalization and leadership track. 2. Business school professorship. After a twenty- two- year hiatus, I returned to academia to hone the methodology described in this book, serving for one year as the Bodily Professor in Analytics at the Darden School of Business at the University of Virginia. The switch in departments— from computer science years ago to business more recently— reflects my shift in focus: For ML to succeed, we need a business- side vantage. 3. More expansive training. Finally, I’ve launched an online course, “Machine Learning Leadership and Practice: End- to- End Mastery,” to broaden the almost universally narrow focus of today’s ML courses— which typically jump straight to the number crunching, forgoing the extensive business planning that should come first. If you don’t have time for a three- month course, you might instead just read this book. It covers the disciplined approach required to deploy ML initiatives, formulated as a six- step playbook that I call