Share E-Book

Applied Machine Learning Explainability Techniques Make ML models explainable and trustworthy for practical applications using… (Aditya Bhattacharya)(Z-Library)

Author

Artificial Intelligence
Language English

Leverage top XAI frameworks to explain your machine learning models with ease and discover best practices and guidelines to build scalable explainable ML systems Key Features: Explore various explainability methods for designing robust and scalable explainable ML systems Use XAI frameworks such as LIME and SHAP to make ML models explainable to solve practical problems Design user-centric explainable ML systems using guidelines provided for industrial applications

Format PDF
Size 17.5 MB
12
Views
0
Downloads
0.00
Total Donations
(First 20 pages)

Registered users can read the full content for free

Register as a Gaohf Library member to read the complete e-book online for free and enjoy a better reading experience.

Page 1
(This page has no text content)
Page 2
Applied Machine Learning Explainability Techniques Copyright © 2022 Packt Publishing All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information. Publishing Product Manager: Dinesh Chaudhary Senior Editor: Tazeen Shaikh Content Development Editor: Manikandan Kurup Technical Editor: Devanshi Ayare Copy Editor: Safis Editing Project Coordinator: Farheen Fathima Proofreader: Safis Editing Indexer: Sejal Dsilva Production Designer: Jyoti Chauhan Marketing Coordinator: Shifa Ansari and Abeer Riyaz Dawe First published: July 2022 Production reference: 2010722 Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK. ISBN 978-1-80324-615-4 www.packt.com
Page 3
Contributors About the author Aditya Bhattacharya is an explainable AI researcher at KU Leuven with 7 years of experience in data science, machine learning, IoT, and software engineering. Prior to his current role, Aditya worked in various roles in organizations such as West Pharma, Microsoft, and Intel to democratize AI adoption for industrial solutions. As the AI lead at West Pharma, he contributed to forming the AI Center of Excellence, managing and leading a global team of 10+ members focused on building AI products. He also holds a master's degree from Georgia Tech in computer science with machine learning and a bachelor's degree from VIT University in ECE. Aditya is passionate about bringing AI closer to end users through his various initiatives for the AI community. I am immensely grateful to all who have been close to me and have supported me throughout the journey of writing this book, especially my wife Shreya, parents, sister, and all my aunts. A special thanks to my colleagues at the Augment research group of KU Leuven. I am grateful to Dr. Katrien Verbert for giving me the opportunity to pursue my journey as an XAI researcher. A big shoutout to all the reviewers for helping me throughout the writing process. Last, but not least, thanks to Abhijit Jana, who has always inspired me to step outside my comfort zone and pursue bigger challenges in life.
Page 4
About the reviewers Sumedh Vilas Datar is a machine learning engineer with 6 years of work experience in the field of deep learning, machine learning, and software engineering. He has a proven track record of single-handedly delivering end-to-end engineering solutions to real-world problems. He works at the intersection of engineering and products and has developed deep learning products from scratch that have been used by a lot of customers. Currently, Sumedh works in R&D on applied deep learning and has several granted patents and several more applied for. Sumedh studied biomedical engineering focused on computer vision and then went on to pursue a master's in computer science focused on AI. Abhijit Jana is a trusted technology leader and advisor with 15 years of experience in the IT industry, with expertise in development, architecting, engineering, consulting, service delivery, and leadership. He is currently associated with West Pharmaceutical Services as a director of software engineering and is responsible for building and leading the software engineering team. Previously, he worked at Microsoft and is a former Microsoft MVP and Code Project MVP, and has been a speaker at various technology conferences. He is the author of the book Kinect for Windows SDK Programming Guide and coauthored the book HoloLens Blueprint. He is also the founder of Daily .NET Tips, a well-known website for developers, architects, and consultants. Sk Nishan Ali works as a full stack data scientist at UnitedHealth Group. He has 6 years of experience across diverse areas of AI and machine learning, which include computer vision, classical machine learning, and natural language processing, and touches upon several business domains: healthcare, CRM, and sourcing. He has been instrumental in building high-performance end-to-end AI/ML products.
Page 5
(This page has no text content)
Page 6
Table of Contents Preface Section 1 – Conceptual Exposure 1 Foundational Concepts of Explainability Techniques Introduction to XAI   4 Understanding the key terms   4 Consequences of poor predictions   6 Summarizing the need for model explainability   8 Defining explanation methods and approaches   11 Dimensions of explainability   12 Addressing key questions of explainability   13 Understanding different types of explanation methods   15 Understanding the accuracy interpretability trade-off   16 Evaluating the quality of explainability methods    18 Criteria for good explainable ML systems   19 Auxiliary criteria of XAI for ML systems   22 Taxonomy of evaluation levels for explainable ML systems   23 Summary   26 References   26 2 Model Explainability Methods Technical requirements    28 Types of model explainability methods   28 Knowledge extraction methods    31 EDA   31 Result visualization methods   36 Using comparison analysis   36 Using Surrogate Explainer methods   37 Influence-based methods   39 Feature importance   40 Sensitivity analysis   41
Page 7
viii Table of Contents PDPs   43 LRP   44 Representation-based explanation   49 VAMs   50 Example-based methods   51 CFEs in structured data   52 CFEs in unstructured data   54 Summary   56 References   56 3 Data-Centric Approaches Technical requirements    60 Introduction to data-centric XAI   61 Analyzing data volume   61 Analyzing data consistency   62 Analyzing data purity   63 Thorough data analysis and profiling process   64 The need for data analysis and profiling processes   65 Data analysis as a precautionary step   65 Building robust data profiles   66 Monitoring and anticipating drifts   67 Detecting drifts   68 Selection of statistical measures   68 Checking adversarial robustness   71 Impact of adversarial attacks   72 Methods to increase adversarial robustness   74 Evaluating adversarial robustness   75 Measuring data forecastability    76 Estimating data forecastability   77 Summary   79 References   80 Section 2 – Practical Problem Solving 4 LIME for Model Interpretability Technical requirements    84 Intuitive understanding of LIME   84 Learning interpretable data representations   85 Maintaining a balance in the fidelity-interpretability trade-off   86 Searching for local explorations   86 What makes LIME a good model explainer?   88 SP-LIME   89 A practical example of using LIME for classification problems   90 Potential pitfalls   97 Summary   98 References   98
Page 8
Table of Contents ix 5 Practical Exposure to Using LIME in ML Technical requirements    100 Using LIME on tabular data   100 Setting up LIME   100 Discussion about the dataset   101 Discussions about the model   102 Application of LIME   102 Explaining image classifiers with LIME   104 Setting up the required Python modules   104 Using a pre-trained TensorFlow model as our black-box model   105 Application of LIME Image Explainers   105 Using LIME on text data   107 Installing the required Python modules   108 Discussions about the dataset used for training the model   108 Discussions about the text classification model    109 Applying LIME Text Explainers   109 LIME for production-level systems   111 Summary   112 References   112 6 Model Interpretability Using SHAP Technical requirements    114 An intuitive understanding of the SHAP and Shapley values   114 Introduction to SHAP and Shapley values   115 What are Shapley values?   115 Shapley values in ML   119 The SHAP framework   120 Model explainability approaches using SHAP   121 Visualizations in SHAP   121 Explainers in SHAP   130 Using SHAP to explain regression models   133 Setting up SHAP   134 Inspecting the dataset   134 Training the model   137 Application of SHAP   138 Advantages and limitations of SHAP   139 Advantages    139 Limitations   140 Summary   141 References   141
Page 9
x Table of Contents 7 Practical Exposure to Using SHAP in ML Technical requirements    144 Applying TreeExplainers to tree ensemble models   144 Installing the required Python modules   145 Discussion about the dataset   145 Training the model   148 Application of TreeExplainer in SHAP   149 Explaining deep learning models using DeepExplainer and GradientExplainer   152 GradientExplainer   153 Discussion on the dataset used for training the model   153 Using a pre-trained CNN model for this example   154 Application of GradientExplainer in SHAP   155 Exploring DeepExplainers   156 Application of DeepExplainer in SHAP   157 Model-agnostic explainability using KernelExplainer   158 Application of KernelExplainer in SHAP   159 Exploring LinearExplainer in SHAP   160 Application of LinearExplainer in SHAP   161 Explaining transformers using SHAP    162 Explaining transformer-based sentiment analysis models   162 Explaining a multi-class prediction transformer model using SHAP   164 Explaining zero-shot learning models using SHAP   166 Summary   169 References   169 8 Human-Friendly Explanations with TCAV Technical requirements    172 Understanding TCAV intuitively   172 What is TCAV?   173 Explaining with abstract concepts   174 Goals of TCAV   174 Approach of TCAV   175 Exploring the practical applications of TCAV   176 Getting started   177 About the data   178 Discussions about the deep learning model used   179 Model explainability using TCAV   180 Advantages and limitations   181 Advantages   182 Limitations   183
Page 10
Table of Contents xi Potential applications of concept-based explanations   184 Summary   186 References   187 9 Other Popular XAI Frameworks Technical requirements    190 DALEX   190 Setting up DALEX for model explainability   192 Discussions about the dataset   192 Training the model   193 Model explainability using DALEX   194 Model-level explanations   195 Prediction-level explanations   198 Evaluating model fairness   201 Interactive dashboards using ARENA   203 Explainerdashboard   205 Setting up Explainerdashboard   205 Model explainability with Explainerdashboard   206 InterpretML   210 Supported explanation methods   211 Setting up InterpretML   211 Discussions about the dataset   212 Training the model   213 Explainability with InterpretML   213 ALIBI   220 Setting up ALIBI   220 Discussion about the dataset   221 Training the model   221 Model explainability with ALIBI   222 DiCE   226 CFE methods supported in DiCE   226 Model explainability with DiCE   227 ELI5   230 Setting up ELI5   230 Model explainability using ELI5   230 H2O AutoML explainers   232 Explainability with H2O explainers   233 Quick comparison guide   235 Summary   237 References   237 Section 3 – Taking XAI to the Next Level 10 XAI Industry Best Practices Open challenges of XAI   243 Guidelines for designing explainable ML systems   246 Adopting a data-first approach for explainability   249 Emphasizing IML for explainability   250
Page 11
xii Table of Contents Emphasizing prescriptive insights for explainability   252 Summary   254 References   254 11 End User-Centered Artificial Intelligence User-centered XAI/ML systems   258 Different aspects of end user-centric XAI   259 Rapid XAI prototyping using EUCA   263 Efforts toward increasing user acceptance of AI/ML systems using XAI   265 Providing a delightful UX   266 Summary   266 References   267 Index Other Books You May Enjoy
Page 12
Preface Explainable AI (XAI) is an emerging field for bringing artificial intelligence (AI) closer to non-technical end-users. XAI promises to make machine learning (ML) models transparent, and trustworthy and promote AI adoption for industrial and research use-cases. This book is designed with a unique blend of industrial and academic research perspectives for gaining practical skills in XAI. ML/AI experts working with data science, ML, deep learning, and AI will be able to put their knowledge to work with this practical guide to XAI for bridging the gap between AI and the end-user. The book provides a hands-on approach for implementation and associated methodologies of XAI that will have you up-and-running, and productive in no time. Initially, you will get a conceptual understanding of XAI and why it's needed. Then, you will get the necessary practical experience of utilizing XAI in the AI/ML problem-solving process by making use of state-of-the-art methods and frameworks. Finally, you will get the necessary guidelines to take XAI to the next step and bridge the existing gaps between AI and end-users. By the end of this book, you will be able to implement XAI methods and approaches using Python to solve industrial problems, address the key pain points encountered, and follow the best practices in the AI/ML life cycle. Who this book is for This book is designed for scientists, researchers, engineers, architects, and managers who are actively engaged in the field of ML and related areas. In general, anyone who is interested in problem-solving using AI would benefit from this book. You are recommended to have a foundational knowledge of Python, ML, deep learning, and data science. This book is ideal for readers who are working in the following roles: • Data and AI scientists • AI/ML engineers • AI/ML product managers
Page 13
xii Preface • AI product owners • AI/ML researchers • User experience and HCI researchers In general, any ML enthusiast with a foundational knowledge of Python will be able to read, understand and apply knowledge gained from this book. What this book covers Chapter 1, Foundational Concepts of Explainability Techniques, gives the necessary exposure to Explainable AI and help you understand it's importance. This chapter covers various terminology and concepts related to explainability techniques, which is frequently used throughout this book. This chapter also covers the key criteria of human- friendly explainable ML systems and different approaches to evaluating the quality of the explainability techniques. Chapter 2, Model Explainability Methods, discusses the various model explainability methods used for explaining black-box models. Some of these are model agnostic, some are model specific. Some of these methods provide global interpretability while others provide local interpretability. This chapter will introduce you to a variety of techniques that can be used for explaining ML models and provides recommendation for the right choice of explainability method. Chapter 3, Data-Centric Approaches, introduces the concept of data-centric XAI. This chapter covers various techniques to explain the working of ML systems in terms of the properties of the data, data volume, data consistency, data purity and actionable insights generated from the underlying training dataset. Chapter 4, LIME for Model Interpretability, covers the application of one of the most popular XAI frameworks, called LIME. This chapter discusses about the intuition behind the working of the LIME algorithm and some important properties of the algorithm which makes the generated explanations human-friendly. Certain advantages and limitations of the LIME algorithm are also discussed in this chapter, along with a code tutorial for applying LIME for a classification problem. Chapter 5, Practical Exposure to Using LIME in ML is an extension of the previous chapter, but more focused towards the practical applications of the LIME Python framework on different types of datasets like images, texts along with structured tabular data. Practical code examples are also covered in this chapter for providing exposure to on-hand knowledge using Python LIME framework. This chapter also covers if LIME is a good fit for production-level ML systems.
Page 14
Preface xiii Chapter 6, Model Interpretability Using SHAP focuses on understanding the importance of the SHAP Python framework for model explainability. It covers the intuitive understanding of Shapley values and SHAP. This chapter also discusses how to use SHAP for model explainability through a variety of visualization and explainer methods. A code walkthrough for using SHAP to explain regression models is also covered in this chapter. Finally, we will discuss the key advantages and limitations of SHAP. Chapter 7, Practical Exposure to Using SHAP in ML provides the necessary practical exposure of using SHAP with tabular structured data as well unstructured data like images and texts. We have discussed about the different explainers available in SHAP for both model-specific and model agnostic explainability. We have applied SHAP for explaining linear models, tree ensemble models, convolution neural network models and even transformer models in this chapter. Necessary code tutorials are also covered in this chapter for providing exposure to hands-on knowledge using Python SHAP framework. Chapter 8, Human-Friendly Explanations with TCAV covers the concepts of TCAV, a framework developed by Google AI. This chapter provides both conceptual understanding of TCAV and practical exposure to applying the Python TCAV framework. The key advantages and limitations of TCAV are discussed along with interesting ideas about potential research problems that can be solved using concept-based explanations are discussed in the chapter.  Chapter 9, Other Popular XAI Frameworks covers about seven popular XAI frameworks available in Python – DALEX, Explainerdashboard, InterpretML, ALIBI, DiCE, ELI5, and H2O AutoML explainers. We have discussed about the supported explanation methods for each of the framework, practical application, and the various pros and cons of each framework. This chapter also provides a quick comparison guide for helping you decide which framework you should go for considering your own use-case. Chapter 10, XAI Industry Best Practices focuses on the best practices for designing explainable AI systems for industrial problems. In this chapter, we have discussed about the open challenges of XAI and necessary design guidelines for explainable ML systems, considering the open challenges. We have also highlighted the importance of considering data-centric approaches of explainability, interactive machine learning and prescriptive insights for designing explainable AI/ML systems. Chapter 11, End User-Centered Artificial Intelligence introduces the ideology of end user centered artificial intelligence (ENDURANCE) for the design and development of explainable AI/ML Systems. We have discussed about the importance of using XAI to steer towards the main goals of the end user for building explainable AI/ML systems. Using some of principles and recommended best practices presented in the chapter, we can bridge the gap between AI and the end user to a great extent!
Page 15
xiv Preface To get the most out of this book To run the code tutorials provided in this book, you will need a Jupyter environment with Python 3.6+. This can be achieved in either of the following ways: • Install one on your machine locally via Anaconda Navigator or from scratch with pip. • Use a cloud-based environment such as Google Colaboratory, Kaggle notebooks, Azure notebooks, or Amazon SageMaker. You can take a look at the supplementary information provided at the code repository if you are new to Jupyter notebooks: https://github.com/PacktPublishing/ Applied-Machine-Learning-Explainability-Techniques/blob/main/ SupplementaryInfo/CodeSetup.md. You can also take a look at https://github.com/PacktPublishing/ Applied-Machine-Learning-Explainability-Techniques/blob/main/ SupplementaryInfo/PythonPackageInfo.md and https://github. com/PacktPublishing/Applied-Machine-Learning-Explainability- Techniques/blob/main/SupplementaryInfo/DatasetInfo.md for getting the supplementary information about the Python packages and datasets used in the tutorial notebooks. For instructions on installing the Python packages used throughout the book, please refer the specific notebook provided in the code repository. For any additional help needed, please refer the original project repository of the specific package. You can use PyPi (https://pypi.org/) and search for the specific package and navigate to the code repository of the project. It is expected that installation or execution instructions of these packages can change from time to time, given how often packages change. We also tested the code with specific versions detailed in the Python package information README file under the supplementary information provided at the code repository. So, if anything doesn't work as expected with the later versions, please install the specific version mentioned in the README instead. If you are using the digital version of this book, we advise you to type the code yourself or access the code from the book's GitHub repository (a link is available in the next section). Doing so will help you avoid any potential errors related to the copying and pasting of code.
Page 16
Preface xv For beginners without any exposure to ML or data science, it is recommended to read the book sequentially as many important concepts are explained in sufficient detail in the earlier chapters. Seasoned ML or data science experts who are relatively new to the field of XAI can skim through the first three chapters to get clear conceptual understanding of various terminology used. For chapters four to nine, any order should be fine for seasoned experts. For all level of practitioners, it is recommended that you read chapter 10 and 11 only after covering all the nine chapters. Regarding the code provided, it is recommended that you either read each chapter and then run the corresponding code, or you can run the code simultaneously while reading the specific chapters. Sufficient theory is also added in the Jupyter notebooks to help you understand the overall flow of the notebook. When you are reading the book, it is recommended that you take notes of the important terminologies covered and try to think of ways in which you could apply the concept or the framework learned. After reading the book and going through all the Jupyter notebooks, hopefully, you will be inspired to apply the newly gained knowledge into action! Download the example code files You can download the example code files for this book from GitHub at https:// github.com/PacktPublishing/Applied-Machine-Learning- Explainability-Techniques. If there's an update to the code, it will be updated in the GitHub repository. We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out! Download the color images We also provide a PDF file that has color images of the screenshots and diagrams used in this book. You can download it here: https://packt.link/DF7lG.
Page 17
xvi Preface Conventions used There are a number of text conventions used throughout this book. Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: "For this example, we will use the RegressionExplainer and ExplainerDashboard submodules." A block of code is set as follows: pdp = PartialDependence(     predict_fn=model.predict_proba,     data=x_train.astype('float').values,     feature_names=list(x_train.columns),     feature_types=feature_types) pdp_global=pdp.explain_global(name='Partial Dependence') When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold: explainer = shap.Explainer(model, x_test) shap_values = explainer(x_test) shap.plots.waterfall(shap_values[0], max_display = 12,                      show=False) Bold: Indicates a new term, an important word, or words that you see onscreen. For instance, words in menus or dialog boxes appear in bold. Here is an example: "Due to these known drawbacks, the search for a robust Explainable AI (XAI) framework is still on." Tips or important notes Appear like this. Get in touch Feedback from our readers is always welcome. General feedback: If you have questions about any aspect of this book, email us at customercare@packtpub.com and mention the book title in the subject of your message.
Page 18
Preface xvii Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata and fill in the form. Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at copyright@packt.com with a link to the material. If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com. Share Your Thoughts Once you've read Applied Machine Learning Explainability Techniques, we'd love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback. Your review is important to us and the tech community and will help us make sure we're delivering excellent quality content.
Page 19
(This page has no text content)
Page 20
Section 1 – Conceptual Exposure This section will give you the necessary conceptual exposure to explainability techniques for machine learning (ML) models with practical examples. You will learn about the foundational concepts, different dimensions of explainability, various model explainability methods, and even data-centric approaches to explainability. Knowledge of the foundational concepts will help you understand the guidelines for designing robust explainable ML systems like those covered in this book. This section comprises the following chapters: • Chapter 1, Foundational Concepts of Explainability Techniques • Chapter 2, Model Explainability Methods • Chapter 3, Data-Centric Approaches
The above is a preview of the first 20 pages. Register to read the complete e-book.

Recommended for You

Loading recommended books...
Failed to load, please try again later

Tip the Site

Scan the WeChat Pay or Alipay code to tip. No login required.

WeChat Pay
Alipay
Back to List