Page
1
(This page has no text content)
Page
2
Praise for Learning AutoML I’ve watched a lot of teams waste enormous energy on ML plumbing that AutoML should have handled. What Kerem Tomak gets right—and most resources miss—is that this isn’t about laziness or shortcuts. It’s about where human judgment actually adds value. In a world where agentic systems compose, retrain, and evaluate models on the fly, understanding AutoML at this depth isn’t optional—it’s foundational. The case studies show what it looks like when that boundary is drawn correctly: faster iteration, better models, and teams that can finally think about the problem instead of the pipeline. —Ashkan Roshanayi, CEO, DataChef What sets Kerem’s work apart is that he never treats AutoML as technology for technology’s sake. Throughout the book—from the ROI analyses and industry-specific business cases in the opening chapters to the case studies where he maps every percentage point of model improvement to concrete financial outcomes like freed capital, recovered revenue, and operational savings— Kerem consistently answers the question that matters most to business leaders: “So what does this mean for my bottom line?” This is a rare and valuable quality in a technical book, and it makes Learning AutoML essential reading not just for data scientists, but for anyone responsible for turning AI investments into actual business results. —Baris Kavakli, CEO, Portera
Page
3
Learning AutoML Automating ML Pipelines with AutoGluon, Leading Frameworks, and Real-World Integration Kerem Tomak
Page
4
Learning AutoML by Kerem Tomak Copyright © 2026 Kerem Tomak. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 141 Stony Circle, Suite 195, Santa Rosa, CA 95401. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional sales department: 800- 998-9938 or corporate@oreilly.com. Acquisitions Editor: Aaron Black Development Editor: Shira Evans Production Editor: Christopher Faucher Copyeditor: Arthur Johnson Proofreader: Andrea Schein Indexer: Krsta Technology Solutions Cover Designer: Susan Brown Cover Illustrator: José Marzan Jr. Interior Designer: David Futato Interior Illustrator: Kate Dullea April 2026: First Edition
Page
5
Revision History for the First Edition 2026-04-03: First Release See http://oreilly.com/catalog/errata.csp? isbn=9798341643185 for release details. The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Learning AutoML, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc. The views expressed in this work are those of the author and do not represent the publisher’s views. While the publisher and the author have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights. 979-8-341-64318-5 [LSI]
Page
6
Foreword by Thomas H. Davenport Automated machine learning is a difficult subject to write about. It’s a relatively easy concept to grasp at the highest level—“Wouldn’t it be great if a computer could automatically create a statistical model to fit my data well and make great predictions?—but difficult to address in detail from both organizational and technical perspectives. In fact, most experts on AutoML are quite technical in their backgrounds and orientations, and aren’t really able to discuss the organizational and economic implications at all. This book is different in that it ably discusses both perspectives on the topic. Kerem Tomak is a senior business executive with a hardcore data science background, and he’s able to bridge the two different domains of AutoML. Nevertheless, I would take his advice about what sections of this book to read given your particular background and approach to this topic. Despite the two worlds of the topic that need to be connected, this is an exciting time to write and read a book on AutoML. Professional data scientists, once wary of AutoML because they thought they could create better models “by hand,” have begun to embrace the technology— particularly for early-stage model exploration. Among nonprofessionals there has also been an exciting set of new technology developments. Some AutoML programs were already pretty easy to use by amateurs, with point- and-click interfaces and integration with business intelligence programs. But now generative AI has enabled not only “vibe coding,” but also “vibe data science.” As Tomak notes in the book, the use of language model prompts to create machine learning models is just the latest step in a long series of technological developments that have enabled greater democratization of data science
Page
7
and machine learning. It’s now possible for non-technical users to issue a short prompt to a language model, upload a dataset and quickly receive not only a predictive model that fits the data, but also some feature engineering, consideration of how best to treat missing values, multiple algorithm explorations, and even a couple of pages of how a manager should address the results of the model for maximum benefit. This book doesn’t devote a lot of attention to generative AI- enabled machine learning, because it’s early days for that technology and minor variations in prompts can yield quite different results. However, Tomak does discuss at various points the important topic of when it is appropriate for amateurs to do data science and when professionals are necessary. And while the technologies he focuses on for AutoML projects are somewhat more difficult to learn than a genAI prompt, they are both more accurate and much easier to use than machine learning programs used to be. You should also be aware that this book is about the more traditional type of machine learning-based AI. Now technically speaking, generative AI is a form of machine learning. But this book is about what I call “analytical AI”— the development of machine learning models that use structured numerical data to produce predictions of other numerical data. This type of AI has been around for much longer than generative AI. If you’ve picked up this book you probably know it’s still quite popular and important, though less well-known to the general public. In fact, I’ve done some recent research suggesting that most of today’s organizations tend to get more economic value from analytical AI than from generative AI. And the value of analytical AI only multiplies when an organization uses
Page
8
AutoML to create and manage machine learning models. Besides, there is no shortage of books on generative AI. So I hope you will read this book and then accelerate and democratize your own and your organization’s use of machine learning with AutoML. You can also manage the resulting models effectively over time and ensure that they are still good predictors of the outcomes you want to predict. There is no better guide to the topic than Kerem Tomak, and he has pulled together all the knowledge you need to become an AutoML expert. Thomas H. Davenport Distinguished Professor, Babson College and Fellow, MIT Initiative on the Digital Economy Author or coauthor of All In on AI, Working with AI, Agentic AI, and Competing on Analytics
Page
9
Foreword by Gregory Wheeler Automated machine learning has been with us, in one form or another, for over a decade. The algorithms are mature. The frameworks are powerful. The cloud platforms compete fiercely for your business. And yet most organizations still struggle to move from a promising model on a laptop to a system that a clinician, a regulator, or a financial analyst can trust. Filling that gap between validation score and earned trust is the central aim of this book. It is well overdue. The gap has persisted because production machine learning is not a harder version of the modeling problem; it is a different problem altogether. A model on a laptop answers a statistical question. A model in production answers to stakeholders, regulators, and the patients or customers whose lives it touches. It must be monitored for drift, audited for fairness, explained to people who have never heard of gradient boosting, nor care to, and then be maintained by teams who did not build it. None of this is algorithmic. Most of it is not, strictly speaking, technical. And yet until now, the AutoML literature has been written almost entirely as if it were. Kerem Tomak brings a rare combination of experience to this subject. He has built production systems, developed products using AutoML in his own company, collaborated with leading lights in analytics and AI, and taught these concepts to diverse audiences, from data science graduate students to tech leads to C-suites. He writes as someone who has sat with the hard problems long enough to know which matter, which to let go, and more importantly, which ones textbooks tend to overlook. The result is a book that follows a model from raw data through hyperparameter optimization and neural architecture search, but then
Page
10
keeps going, through Airflow DAGs and Kubeflow pipelines, through CI/CD for machine learning, through monitoring and drift detection in production. Kerem understands that getting a model into production is not the end of the story. It is just the beginning. The three case studies that close the book bring everything together. A bank processing fifty million transactions a day under hundred-millisecond latency constraints. A retailer forecasting demand across hundreds of stores and tens of thousands of SKUs. A hospital system predicting patient readmission while navigating HIPAA compliance and algorithmic fairness across demographic groups. These are not toy examples. They are the kind of problems that reveal whether you have understood AutoML deeply enough to deploy it responsibly. The healthcare case study deserves particular attention. Tomak walks the reader through three successive approaches to bias mitigation—removing protected attributes, adversarial debiasing, and finally a fairness- aware ensemble with post-hoc calibration—showing concretely why the naïve solutions fail. The demonstration that removing race from the feature set does not prevent proxy discrimination through ZIP codes and insurance type is presented with a clarity and practicality that will stay with you. In regulated industries, the distance between knowing that proxy discrimination exists and knowing how to detect and mitigate it in a production pipeline is the difference between a conference paper and a deployed system. Whether you are a data scientist looking to deepen your implementation skills, a domain expert building your first pipeline, or an engineer tasked with getting AutoML into production, you will find here a guide that respects both
Page
11
the difficulty of the problem and the intelligence of the reader. I am glad Kerem wrote it, and I am glad you are about to read it. Gregory Wheeler Professor of Computational Science & Philosophy Frankfurt School of Finance & Management February 2026
Page
12
Preface Why I Wrote This Book Throughout my career spanning decades in data science and analytics, I’ve witnessed a remarkable transformation in how organizations approach machine learning. What once required teams of PhD-level experts and months of painstaking manual work can now be accomplished by domain experts in days or even hours. This democratization of machine learning capabilities through AutoML represents one of the most significant shifts I’ve observed in the field. Yet despite AutoML’s growing importance, I noticed a critical gap in available resources. Most AutoML documentation focuses on specific tools or provides high- level overviews without practical implementation guidance. Academic papers dive deep into algorithmic details but often lack real-world context. What was missing was a comprehensive resource that bridges theory and practice— one that explains not just how AutoML works but also when and why to use different approaches and how to integrate them into actual business workflows. This gap became particularly apparent in my work within organizations across finance, healthcare, retail, and tech. Time and again, I encountered talented professionals who deeply understood their domain problems but struggled to navigate the AutoML landscape effectively. They needed guidance on selecting the right tools, understanding the trade-offs between different approaches, and implementing solutions that would hold up in production—especially in
Page
13
regulated industries, where explainability, trust, and governance matter. I gained much of the hands-on experience behind this book while experimenting with the tools it covers and using AutoGluon to build a product for my company, med- essence.de. I found AutoGluon to be a fit for our purpose and used it to put one of our use cases in production. While I was able to gather a lot of information from the internet and various papers and books, I also had to learn from GitHub pages, fragmented resources, and sometimes incomplete documentation of open source software through testing and validation. This book grew out of that multiyear effort. My recent collaboration with Thomas Davenport and Ian Barkin on a topic exploring how AI tools enable “citizen developers” further reinforced this idea. We observed that while technology is rapidly democratizing AI development, the gap between available tools and practical knowledge remains substantial. Organizations need more than just access to AutoML platforms—they need a clear understanding of how to use these tools strategically and responsibly. How these citizen-developed solutions make their way into production is also a gray area. This book represents my attempt to fill that knowledge gap. Drawing from years of hands-on experience implementing analytics solutions, developing products using traditional and AutoML tools like AutoGluon and TPOT, teaching these concepts to diverse audiences, and observing what works (and what doesn’t) in real-world deployments, I’ve crafted a resource that serves both newcomers seeking to understand AutoML fundamentals and experienced practitioners looking to deepen their implementation skills.
Page
14
Rather than advocating for any particular tool or approach, this book provides a balanced perspective on the AutoML ecosystem. You’ll find detailed coverage of core concepts, such as hyperparameter optimization and neural architecture search, alongside practical tutorials using AutoGluon, one of the most capable and accessible AutoML frameworks available today. Most importantly, you’ll gain the knowledge needed to make informed decisions about when and how to apply AutoML in your specific context. Who Should Read This Book This book is designed for anyone who wants to understand and effectively apply automated machine learning, regardless of their current technical background. The content is structured to serve multiple audiences simultaneously, with different readers likely to focus on various sections based on their needs and experience levels. Data scientists and ML engineers will find comprehensive coverage of AutoML’s technical foundations, comparative analysis of different optimization approaches, and advanced techniques for customizing automated workflows. Even if you’re already experienced with traditional machine learning, this book will help you understand how AutoML can accelerate your work and expand your capabilities. The sections on hyperparameter optimization, neural architecture search, and production integration provide depth that goes beyond typical tool documentation. Business analysts and domain experts represent the most important audience for this book. If you understand your organization’s data and business problems but lack extensive programming experience, the hands-on sections
Page
15
using AutoGluon will enable you to build sophisticated models with minimal code. The industry-specific examples and case studies will help you identify opportunities to apply AutoML in your domain, while the conceptual explanations ensure you understand what’s happening under the hood. Software engineers and application developers who need to integrate machine learning capabilities into their applications will benefit from the production-focused sections covering deployment, CI/CD, and MLOps integration. You don’t need to become a data science expert to effectively leverage AutoML. Still, you do need to understand how these systems work and how to integrate them reliably into larger software systems. Students and educators in data science, computer science, or related fields will find this book serves as both a comprehensive introduction to AutoML concepts and a practical guide for hands-on learning. The progression from fundamentals through advanced applications, combined with real-world projects and case studies, makes it suitable for both self-directed learning and classroom use. Business leaders and decision makers should focus on the foundational chapters and industry application sections to understand the strategic implications of AutoML. While you may not implement solutions directly, understanding AutoML’s capabilities and limitations is crucial for making informed decisions about AI investments and team development. Consultants and solution architects working with multiple organizations will appreciate the broad coverage of different AutoML approaches, the comparative analysis of tools and techniques, and the industry-specific guidance.
Page
16
This book provides the knowledge base needed to recommend appropriate solutions across diverse client contexts. The book assumes basic familiarity with data analysis concepts but doesn’t require deep machine learning expertise. Mathematical concepts are explained intuitively, with technical details provided for those who want them. Code examples use Python and focus on practical implementation rather than complex programming concepts. Whether you’re looking to automate existing machine learning workflows, explore new applications of AI in your organization, or simply understand how AutoML is transforming the data science landscape, this book provides the knowledge and practical guidance you need to succeed.
Page
17
How This Book Is Organized This book is structured as a comprehensive journey, spanning from AutoML fundamentals to advanced implementation and production deployment. The five main parts build upon each other systematically to take you from theory to practice. However, experienced readers may choose to focus on specific sections based on their immediate needs.
Page
18
Part I: Foundations of AutoML Part I establishes the conceptual groundwork for the rest of the book: Chapter 1 introduces the core concepts of AutoML, explores why it matters for modern organizations, and examines its role in democratizing AI development. Chapter 2 traces the evolution of AutoML through three distinct generations, from early academic tools to today’s enterprise-ready platforms. Chapter 3 introduces the end-to-end AutoML pipeline, providing a complete roadmap from raw data to a deployed model. This chapter serves as the organizing framework for the technical deep dives that follow.
Page
19
Part II: Core AutoML Techniques Part II dives deep into the algorithms and methods that power modern AutoML systems, following the logical flow of the pipeline introduced in Part I: Chapter 4 examines automated feature engineering and data preprocessing, which are crucial capabilities that often determine the success of models in real-world applications. Chapter 5 provides comprehensive coverage of hyperparameter optimization (HPO), the foundational tuning technique underlying virtually all AutoML tools. Chapter 6 explores neural architecture search (NAS), showing how modern systems can automatically design optimal neural network architectures for complex tasks. Together, these chapters provide the technical foundation necessary to understand how AutoML operates under the hood and to make informed decisions about customizing automated workflows.
Page
20
Part III: AutoML for Different Data Types Part III transitions from theory to practice through extensive tutorials using AutoGluon, one of the most capable and accessible AutoML frameworks available: Chapter 7 introduces AutoGluon through tabular data, beginning with installation and environment setup before moving into hands-on modeling The following chapters then focus on specific data types and use cases: Chapter 8 covers text and NLP applications. Chapter 9 addresses time series forecasting and analysis. Chapter 10 explores computer vision applications. The progression from single-modal to multimodal problems reflects the increasing sophistication of modern AutoML capabilities. Each chapter includes both conceptual explanations and complete, runnable code examples that you can adapt to your projects.