Continuous Machine Learning with Kubeflow Performing Reliable MLOps with Capabilities of TFX, Sagemaker and Kubernetes (Aniruddha Choudhury)（Z-Library）

(This page has no text content)

Continuous Machine Learning with Kubeflow Performing Reliable MLOps with Capabilities of TFX, Sagemaker and Kubernetes Aniruddha Choudhury www.bpbonline.com

FIRST EDITION 2022 Copyright © BPB Publications, India ISBN: 978-93-89898-507 All Rights Reserved. No part of this publication may be reproduced, distributed or transmitted in any form or by any means or stored in a database or retrieval system, without the prior written permission of the publisher with the exception to the program listings which may be entered, stored and executed in a computer system, but they can not be reproduced by the means of publication, photocopy, recording, or by any electronic and mechanical means. LIMITS OF LIABILITY AND DISCLAIMER OF WARRANTY The information contained in this book is true to correct and the best of author’s and publisher’s knowledge. The author has made every effort to ensure the accuracy of these publications, but publisher cannot be held responsible for any loss or damage arising from any information in this book. All trademarks referred to in the book are acknowledged as properties of their respective owners but BPB Publications cannot guarantee the accuracy of this information.

www.bpbonline.com

Dedicated to My beloved parents and family

About the Author Aniruddha Choudhury has 5 years of IT professional experience in providing Artificial Intelligence development solutions, MLOPS Kubeflow, Multi Cloud GCP, AWS, Azure and is passionate about providing Data Science and Data Engineering complex solutions in machine learning and deep learning. He is always looking for new opportunities for a new dimensional challenge for high impact business problems to become a valuable contributor for his future employers. Presently he is working in Publicis Sapient as Senior Data scientist (Full stack MLOPS) for the last 2 year. Previously he worked with Incture technology and before that he has worked at Wells Fargo Bank on diverse financial products’ AI solutions on various lines of business. As a tech geek, Aniruddha is always enthusiastic about working in cross-dimensional knowledge. He is working on Kaggle and Google data projects building a statistical and machine learning NLP model with an end-to-end data wrangling and preparation, building model framework with visualization and predictive/text/image analytics and Amazon AWS and Microsoft Azure DataBricks Cloud with Apache scala and spark and finding patterns in all forms of data. He has a passion to break complex problems in data science field and find resolutions with deep learning and machine learning. He is also working on deep learning frameworks like Tensorflow, Keras, and Pytorch. As an individual, Aniruddha always believes in constantly learning new skills and taking the road less

travelled. He likes to keep himself updated about the technological world and is always toying with some new innovation. He has mastered in building Artificial Intelligence solutions and finding complex patterns from research papers to gain optimal solution to current product development and possesses a self- innovative mind alongside.

About the Reviewer Rajdeep Kumar is a lead data scientist at one of the top consulting companies. He has a Post Graduate Degree in Data Science from BITS Pilani, and is an alumni of IIIT Bangalore. He has more than 11 years of work experience in the IT industry. In his capacity, he drives and contributes to the Data Science activities along with defining the road map, scoping and mentoring the team members. He is an expert in deriving and productionizing end-to-end ML solutions to grow business and have a positive impact on the key business KPIs.

Acknowledgement There are a few people I want to thank for the continued and ongoing support they have given me during the writing of this book. First and foremost, I would like to thank my parents for continuously encouraging me for writing the book — I could have never completed this book without their support. I would like to thank my friend Dr. Someswar Deb, who is working as MSL in Novartis, for his constant support and motivation. I am grateful to the journey provided from my companies who gave me support throughout the learning process of my career. Thank you for all the hidden support provided. I gratefully acknowledge Mr. Abhishek Kumar, Senior Director of Public is Sapient for his kind technical help for deployment related stuffs which was helpful for this book. I would like to thank Sivaram Annadurai Senior Manager at Publicis Sapient for his constant motivation in technical and business aspects in machine learning real-time projects. My gratitude also goes to the team at BPB Publications for being supportive enough to provide me quite a long time to finish the first part of the book and also allow me to publish the book in multiple parts. Since image processing being a vast and very active area of research, it was impossible to deep-dive into different classes of problems in a single book, especially by not making it too voluminous.

Preface This book focuses on the DevOps and MLOps of deploying and productionising machine learning projects with Kubeflow in Google Cloud platform. The authors feel that in this era of machine learning, lot of companies failed to make production of AI/ML projects in real time which was also a study from Forbes. It is compelling and relevant content for today’s practicing DevOps/MLOps teams as this sector is still changing. So, many machine learning platforms today take different approaches to the architecture and solution space of managing machine learning workflows. The core concepts of Kubernetes and Kubeflow and its architecture alongside teaches us how to approach and make your AI/ML projects from training to serving with scale in production with Kubeflow. This book starts by taking you through today’s machine learning infrastructure of Kubernetes and Kubeflow architecture. We then go on to outline the core principles of deploying various AI/ML use cases with TensorFlow training serving with Kubeflow and explain how Kubernetes solves some of the issues that arise. We further show how to use TFX with Kubeflow alongside Explainable AI for determining fairness and biasness with What-if Tool. We learn various serving techniques framework for different use cases with Kubeflow KF serving. After that we look at building sample computer vision based UI in streamlit and deploying that in Google cloud platform Kubernetes and Heroku deployment.

This book is divided into 8 chapters. They cover Kubernetes, Kubeflow basics, advance deployment projects with Kubeflow, AWS Sagemaker deployment and explainable AI with real time examples for deployment and container creation with Docker and building pipeline in Kubeflow. More interest will arise among learners in Machine learning deployment with Kubeflow. The details are listed as follows: Chapter 1: In this chapter, we will learn about the complete features of Kubeflow, how it works and its need. We will also learn about the architecture functionality of Kubernetes such as service, pod, Ingress, and so on. We will learn to build the docker image and learn it’s working. Here, we will see the components of Kubeflow advantage, which we will be using in the upcoming chapters. Then, we will proceed towards the complete setup of Kubeflow in the Google Cloud platform and Jupyter notebook setup. We have an optional item – how to create the persistent volume claim and attach it to the file store to save your codes and data. Chapter 2: In this chapter, we will build an end-to-end TensorFlow classification model deployment with Kubeflow orchestration which includes deploying Kubeflow in Kubernetes Cluster in GCP, building the pipeline components for the model with Docker and Kubeflow SDK, and then serving the model with KF serving to have an endpoint for prediction. We will also track the monitoring and performance for our serving traffic endpoint in Grafana dashboard.

Chapter 3: In this chapter, we will build an end-to-end TensorFlow computer vision model with OpenCV operation and deploy that with the Kubeflow orchestration, which includes deploying Kubeflow in Kubernetes cluster in GCP, building the pipeline components for the model with Docker and Kubeflow SDK and then serving the model with KF serving to have an endpoint for prediction. We will then track the monitoring and performance in Grafana dashboard. Chapter 4: In this chapter, we will build an end-to-end structured data classification model and make it ready for production with the help of TFX, and serve the model outputs with TF serving to get the prediction. We will also be building the TensorFlow ecosystem model and visualizing the evaluation with Tensorboard and Fairness. Then, we will learn about the various TFX components like TFT, TFMA, TFDV, and so on. Later on, we will create a Kubeflow Pipeline on Google Cloud. Chapter 5: In this chapter, we will work on a classification model with the hotel booking dataset, train the TensorFlow and boosting models, and visualize the advanced explanation of our model results with Tensorboard, Shap, and What-if products. Chapter 6: In this chapter, we will build an end-to-end Light Model framework and will monitor the model performance in the Weights and Biases (Wandb) tool. Within Weights and Biases, we will see the live model RMSE graphs and parallel coordinates’ hyper parameter performance graphs for each iteration. Next, we will deploy the model with the KF serving in our Kubernetes Cluster inside Google Cloud platform. We will be serving model endpoint which will be used for prediction and monitored in the

Grafana Dashboard, such as model rate request with respect to the time and CPU and GPU consumption. Chapter 7: In this chapter, we will work on the Housing Price Sales dataset project, where we will completely run, evaluate, and deploy the model in the Amazon SageMaker Cloud environment and use S3 for data storage. We will also be using the in-built container algorithm XG-Boost for model building so that we are able to understand the architecture of SageMaker model building framework end to end. Chapter 8: In this chapter, we will build an end-to-end web application for the computer vision models, and build the UI with Streamlit. We will be learning about many Open CV models for image like cropping, changing pixels, and so on. Next, we will host the web application with the Heroku Container Registry or Kubernetes Cluster as a service application in Google Cloud.

Downloading the code bundle and coloured images: Please follow the link to download the Code Bundle and the Coloured Images of the book: https://rebrand.ly/bfbaf6 Errata We take immense pride in our work at BPB Publications and follow best practices to ensure the accuracy of our content to provide with an indulging reading experience to our subscribers. Our readers are our mirrors, and we use their inputs to reflect and improve upon human errors, if any, that may have occurred during the publishing processes involved. To let us maintain the quality and help us reach out to any readers who might be having difficulties due to any unforeseen errors, please write to us at : errata@bpbonline.com Your support, suggestions and feedbacks are highly appreciated by the BPB Publications’ Family.

Did you know that BPB offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.bpbonline.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at business@bpbonline.com for more details. At you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on BPB books and eBooks.

BPB is searching for authors like you If you're interested in becoming an author for BPB, please visit www.bpbonline.com and apply today. We have worked with thousands of developers and tech professionals, just like you, to help them share their insight with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea. The code bundle for the book is also hosted on GitHub at In case there's an update to the code, it will be updated on the existing GitHub repository. We also have other code bundles from our rich catalog of books and videos available at Check them out! PIRACY If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at business@bpbonline.com with a link to the material. If you are interested in becoming an author

If there is a topic that you have expertise in, and you are interested in either writing or contributing to a book, please visit REVIEWS Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions, we at BPB can understand what you think about our products, and our authors can see your feedback on their book. Thank you! For more information about BPB, please visit

Table of Contents 1. Introduction to Kubeflow & Kubernetes Cloud Architecture Structure Objectives 1.1 Docker understanding 1.1.1 Dockerfile 1.2 Kubernetes Architecture 1.2.1 What is Kubernetes? 1.2.2 Why do we need Kubernetes? 1.2.3 What are the Advantages of Kubernetes? 1.2.4 How do Kubernetes work? 1.3 Kubernetes components 1.3.1 Types of Services 1.4 Introduction on Kubeflow Orchestration for ML Deployment 1.5 Components of Kubeflow 1.5.1 Central Dashboard 1.5.2 Registration Flow 1.5.3 Metadata 1.5.4 Jupyter Notebook server 1.5.5 Katib 1.6 Getting Started in GCP Kubeflow setup 1.6.1 Install and Set Up kubectl 1.6.2 Install and Set Up gcloudsdk 1.6.3 Set Up OAuth from Cloud IAP 1.6.4 Set Up Docker 1.6.5 Set Up Kubeflow in Kubernetes Cluster in GCP 1.6.6 Connect to cluster and Deploy Grafana 1.6.7 Jupyter Notebook server setup in Kubeflow

1.7 Optional: PVC setup for Jupyter Notebook 1.8 Conclusion 1.9 Reference 2. Developing Kubeflow Pipeline in GCP Structure Objectives 2.1 Problem statement 2.2 Getting started in GCP Kubeflow setup 2.3 Breakdown technique to build production pipeline 2.4 Building the Kubeflow Pipeline components for TensorFlow model 2.3.1 Data Extraction or Ingestion Component 2.3.2 Data pre-processing component 2.3.3 Training model component 2.3.4 Evaluation component 2.5 Serving the Model with KF Serving 2.6 Building the pipeline end to end 2.7 Monitoring the performance with Grafana dashboard 2.8 Conclusion 2.9 Reference 3. Designing Computer Vision Model in Kubeflow Structure Objectives 3.1 Problem statement 3.2 Getting started in GCP Kubeflow setup 3.3 Analytics behind the problem statement 3.4. Building the Kubeflow pipeline components for Computer Vision (CNN) TensorFlow model

3.4.1 Data extraction or Ingestion component 3.4.2 Data pre-processing component 3.4.3 Training model component 3.3.4 Evaluation component 3.5. Serving the Model with KF Serving 3.6 Building the pipeline end to end 3.7. Auto-Scaling of the Serving Endpoint 3.8 Conclusion 3.9 Reference 4. Building TFX Pipeline Structure Objective 4.1 Problem statement 4.2 Architecture of TFX components 4.3 TFX environment setup 4.4 TFX pipeline components 4.4.1 ExampleGen 4.4.2 StatisticsGen 4.4.3 SchemaGen 4.4.4 ExampleValidator 4.4.5 Transform 4.4.6 Tuner and Trainer 4.4.7 Evaluator 4.4.7.1 Fairness and TFMA Visualization 4.4.8 Pusher 4.5 Serve the Model with TF Serving 4.6 Building Kubeflow Pipeline Orchestrator 4.7 Conclusion 4.8 Reference

Statistics

Uploader

Continuous Machine Learning with Kubeflow Performing Reliable MLOps with Capabilities of TFX, Sagemaker and Kubernetes (Aniruddha Choudhury)（Z-Library）

Tags

Text Preview (First 20 pages)

Registered users can read the full content for free

Comments 0

Blog & Notes

Recommended for You

Statistics

Uploader

Continuous Machine Learning with Kubeflow Performing Reliable MLOps with Capabilities of TFX, Sagemaker and Kubernetes (Aniruddha Choudhury)（Z-Library）

Tags

Text Preview (First 20 pages)

Registered users can read the full content for free

Comments 0

Reply to Comment

Edit Comment

Blog & Notes

Recommended for You