Python Machine Learning Projects Learn how to build Machine Learning projects from scratch (Dr. Deepali R Vora, Dr. Gresha S Bhatia) (z-library.sk, 1lib.sk, z-lib.sk)

Dr. Deepali R Vora Dr. Gresha S Bhatia www.bpbonline.com Python Machine Learning Projects Learn how to build Machine Learning projects from scratch

ii  Copyright © 2023 BPB Online All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor BPB Online or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book. BPB Online has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, BPB Online cannot guarantee the accuracy of this information. First published: 2023 Published by BPB Online WeWork 119 Marylebone Road London NW1 5PU UK | UAE | INDIA | SINGAPORE ISBN 978-93-89898-262 www.bpbonline.com

 iii Dedicated to To my parents, in-laws, daughter, husband Mr. Rahul Vora, and family members — Dr. Deepali R Vora To my parents, in-laws, husband Mr. Sachin S Bhatia, and family members — Dr. Gresha S Bhatia

iv  About the Authors • Dr. Deepali R Vora is a Professor and Head of Computer Science & Engineering at Symbiosis Institute of Technology, Pune, and has completed her Ph.D. in Computer Science and Engineering from Amity University, Mumbai. She has more than 22 years of teaching, research as well as Industrial experience. She has more than 60 research papers published in Journals and Conferences of International and national repute. She has co-authored three books and delivered various talks in Data Science and Machine learning. She has conducted hands-on sessions in Data Science using Python for students and faculty. She was appointed as a Syllabus Revision Committee member with Mumbai University and developed the course content for B.E. (Information Technology) course. She has received grants for conducting research and organizing training courses for faculties. She acts as a technical advisor and reviewer for many International Conferences and Journals. Her blogs on KnowledgeHut have received wide acknowledgment. She has developed a course on “Deep Learning” on the Unschool platform. • Dr. Gresha S Bhatia is the Deputy Head of the Computer Engineering Department at Vivekanand Education Society’s Institute of Technology (VESIT), Mumbai, and has completed her Ph.D. Technology from the University of Mumbai. She has more than 25 years of industry and teaching experience. She has published more than 50 research papers in International Journals, conferences, and national conferences of repute. She has been awarded Microsoft AI for Earth Grant in the year 2019 as well as Minor Research Grant from the University of Mumbai. She has authored a book and has delivered sessions on Machine learning and Social Computing. She has also been a member of the Syllabus Revision Committee of Mumbai University for the undergraduate and postgraduate programs in Engineering. She has also been a technical advisor and reviewer for the number of International Conferences and Journals.

 v About the Reviewer Ravi Choudhary is a Data Scientist from New Delhi, India. He has several years of experience working for the Government of India and Top Global companies. Ravi holds a Master’s degree in Artificial Intelligence from IIT and applies his expertise in this field to his work in data science. He is an expert in machine learning and has extensive experience developing and implementing advanced algorithms and models to extract insights from large and complex datasets. He has a deep understanding of various machine learning techniques, such as supervised and unsupervised learning, deep learning, and natural language processing. Ravi has applied his expertise in machine learning to a wide range of industries, including finance, healthcare, and transportation. His work has led to significant improvements in efficiency, accuracy, and performance. He is also known for his ability to communicate complex technical concepts to non-technical stakeholders and effectively collaborate with cross-functional teams.

vi  Acknowledgements We would like to take this opportunity to thank a few people for their continued support while writing this book. First and foremost, we would like to thank our parents and parents-in-law for encouraging us to author the book. This journey was possible due to the constant support of our life partner, who has been our pillar of strength. We are grateful to the BPB publishing house, which showed us the path and initiated our learning process. Our sincere gratitude to all our family members and well-wishers for always standing by us. We sincerely thank all those who have directly or indirectly supported us while authoring this book.

 vii Preface This book covers different aspects of machine learning, and the importance of developing machine learning algorithms that can be applied to real-life projects. The book elaborates on the need and different types of machine learning algorithms. It shows how the right amount of data, and the correct amount of training can help build optimized systems. These optimized systems can help reduce the cost as well as training errors. This book further provides a number of questions to be solved that gives a practical approach. It further gives importance for understanding the python code, its basic concepts as well as the implementation code for machine learning projects. This book is divided into five chapters. The book elaborates on the need for machine learning, its challenges and limitations. It further explores the development of machine learning algorithms through use of Python code. The working of the machine learning algorithms is emphasized through real-life case studies. The best algorithm that can be applied to a case study is elaborated through the various optimization and meta-heuristic approaches. The details of the chapters are specified below. Chapter 1: Introduction to ML – introduces the terms artificial intelligence, data science and machine learning and the differences between them. The various models, the types of machine learning algorithms, and the challenges and limitations faced are further explored. The chapter will further dwell on the working and the application areas of machine learning. Chapter 2: Python Basics for ML – will cover the concepts behind the Python programming language. The syntax and the need for python programming is further emphasized. It will elaborate on python tools and libraries. It further handles the concepts of file and exception handling, which form the crux of the machine learning domain. Chapter 3: An Overview of ML Algorithms – will introduce the various Machine learning algorithms. The major focus is to provide familiarity with the machine learning programs that can learn from the data provided and improve from experience, without any human intervention. These learning tasks may include learning the function that maps the input to the outputs or learning through the hidden structure in unlabeled data using algorithms coined under supervised and

viii  unsupervised learning. To further understand the core concepts, this chapter will explore the working process of any given machine learning algorithm through a number of examples. As Python forms a handy tool to get started with supervised and unsupervised learning, this chapter will explore its various functionalities. The chapter will further elaborate on the performance measures to be considered for evaluating the machine learning system that would have been developed. Chapter 4: Case Studies and Projects in Machine Learning – will provide an insight into the various case studies that use the concepts and algorithms of machine learning. This chapter will further introduce recommendation systems, its needs, the process of generating the recommendation systems as well as the current systems incorporating recommendation systems. Another case study will be focused on the application of machine learning algorithms on text mining applications, opinion mining and sentiment analysis. This chapter will further elaborate on applying various machine learning algorithms to image processing applications. More case studies incorporating predictive analysis, Social Media Analytics, Customer churning analytics, and Analytics in Education Systems will be explored in detail. Chapter 5: Optimization in ML Algorithm – will focus on the improvements in training the algorithms for the most accurate outputs. The chapter will thus focus on determining the best element or path, or techniques to improve the efficiency of algorithms by decreasing the training time as well as the cost incurred for the same. The concept of optimization and the hybrid algorithms that can be utilized will then be explained. The need for optimization techniques in Machine Learning based projects will then be highlighted. This will lead to applying techniques for optimization. The basic optimization techniques will then be explored, and the current research area on meta-heuristic approach will be further elaborated. To conclude the chapter, the various python libraries that could be available for optimization in Machine Learning projects will be explained.

 ix Code Bundle and Coloured Images Please follow the link to download the Code Bundle and the Coloured Images of the book: https://rebrand.ly/sn6fi4n The code bundle for the book is also hosted on GitHub at https://github.com/ bpbpublications/Python-Machine-Learning-Projects. In case there’s an update to the code, it will be updated on the existing GitHub repository. We have code bundles from our rich catalogue of books and videos available at https://github.com/bpbpublications. Check them out! Errata We take immense pride in our work at BPB Publications and follow best practices to ensure the accuracy of our content to provide with an indulging reading experience to our subscribers. Our readers are our mirrors, and we use their inputs to reflect and improve upon human errors, if any, that may have occurred during the publishing processes involved. To let us maintain the quality and help us reach out to any readers who might be having difficulties due to any unforeseen errors, please write to us at : errata@bpbonline.com Your support, suggestions and feedbacks are highly appreciated by the BPB Publications’ Family. Did you know that BPB offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.bpbonline.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at : business@bpbonline.com for more details. At www.bpbonline.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on BPB books and eBooks.

x  Piracy If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at business@bpbonline.com with a link to the material. If you are interested in becoming an author If there is a topic that you have expertise in, and you are interested in either writing or contributing to a book, please visit www.bpbonline.com. We have worked with thousands of developers and tech professionals, just like you, to help them share their insights with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea. Reviews Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions. We at BPB can understand what you think about our products, and our authors can see your feedback on their book. Thank you! For more information about BPB, please visit www.bpbonline.com. Join our book's Discord space Join the book's Discord Workspace for Latest updates, Offers, Tech happenings around the world, New Release and Sessions with the Authors: https://discord.bpbonline.com

 xi Table of Contents 1. Introduction to ML ........................................................................................ 1 Introduction .............................................................................................................. 1 Structure .................................................................................................................... 1 Objectives .................................................................................................................. 2 Introduction to Machine Learning (ML) .............................................................. 2 Models in Machine Learning .................................................................................. 3 Supervised machine learning model through training .....................................3 Unsupervised machine learning model: Self-sufficient learning .....................3 Semi-supervised machine learning model ........................................................4 Reinforcement machine learning model: Hit and Trial ...................................4 Types of Machine Learning Algorithms ............................................................... 5 Working of Machine Learning algorithm ............................................................. 6 Challenges for Machine Learning Projects ........................................................... 7 Limitations of machine learning ............................................................................ 8 Application areas of ML .......................................................................................... 9 Difference between the terms data science, data mining, machine learning and deep learning ................................................................... 11 Conclusion .............................................................................................................. 12 Questions and Answers ......................................................................................... 12 2. Python Basics for ML ....................................................................................23 Introduction ............................................................................................................ 23 Structure .................................................................................................................. 24 Objectives ................................................................................................................ 24 Spyder IDE .............................................................................................................. 24 Jupyter Notebook ................................................................................................... 26 Python: Input and Output Commands ............................................................... 27 Logical Statements ................................................................................................. 29 Loop and Control Statements ............................................................................... 31 Functions and Modules ......................................................................................... 32 Class Handling ........................................................................................................ 35 Exception Handling ............................................................................................... 36 File Handling .......................................................................................................... 36

xii  String functions ...................................................................................................... 38 Conclusion .............................................................................................................. 40 Questions and Answers ......................................................................................... 41 3. An Overview of ML Algorithms ...................................................................53 Introduction ............................................................................................................ 53 Structure .................................................................................................................. 54 Objectives ................................................................................................................ 55 Machine learning modeling flow ......................................................................... 55 Terms used in preprocessing ............................................................................57 Raw data (or just data) ..............................................................................57 Prepared data .............................................................................................57 Need for Data Preprocessing ...........................................................................57 Preprocessing in ML .............................................................................................. 58 Researching the best model for the data .........................................................58 Training and testing the model on data .........................................................60 Evaluation ..................................................................................................61 Hyperparameter tuning ..................................................................................62 Prediction ...................................................................................................62 Metrics Used ...............................................................................................62 Regression algorithms ........................................................................................... 62 Types of regression techniques .........................................................................62 Linear Regression .......................................................................................63 Logistic Regression .....................................................................................63 Polynomial Regression ...............................................................................64 Stepwise Regression ....................................................................................64 Ridge Regression .........................................................................................64 Lasso Regression .........................................................................................64 ElasticNet Regression .................................................................................64 Classification ........................................................................................................... 64 Terminology used in Classification Algorithms .............................................65 Types of classification algorithms ...................................................................65 Performance measures for classification ........................................................67 Clustering ................................................................................................................ 67 Clustering algorithms ......................................................................................68 K-Means Clustering ...................................................................................68

 xiii Mean-Shift Clustering ................................................................................68 Agglomerative Hierarchical Clustering .....................................................69 Clustering Validation ......................................................................................70 Neural Network and SVM .................................................................................... 71 Building Blocks: Neurons ................................................................................71 Combining Neurons into a Neural Network ..................................................72 Training the neural network ...........................................................................72 Neural Network Architectures ........................................................................73 Support vector machine (SVM) ........................................................................... 75 Machine Learning Libraries in Python ............................................................... 77 Numpy .............................................................................................................77 Pandas .............................................................................................................81 Populating the Dataframe .........................................................................82 Displaying the Data Using Dataframe ......................................................83 Accessing the Data Selectively in Dataframe ............................................83 Basic Descriptive Statistics using Pandas ..................................................85 Data transformation ..................................................................................86 Data Preprocessing – Handling missing values .........................................86 MatplotLib .......................................................................................................87 Line Graph ..................................................................................................87 Scatter Plot .................................................................................................89 Bar plot .......................................................................................................90 Histogram ...................................................................................................91 Pie Chart ....................................................................................................92 Evaluation of ML Systems ..................................................................................... 93 Test and Train Datasets ..................................................................................93 Performance Measure .....................................................................................94 Model Evaluation Techniques .........................................................................94 Model evaluation metrics ................................................................................94 Classification Metrics .................................................................................95 Regression Metrics ......................................................................................96 Conclusion .............................................................................................................. 97 Questions ................................................................................................................. 97 4. Case Studies and Projects in Machine Learning ........................................125 Introduction ..........................................................................................................125

xiv  Structure ................................................................................................................126 Objectives ..............................................................................................................127 Recommendation Generation ............................................................................127 Importance of recommendation systems ......................................................127 Key terms used .........................................................................................128 Items/Documents .....................................................................................128 Query/Context .........................................................................................128 Approaches for building recommendation systems ......................................128 Basic recommendation systems ....................................................................130 Candidate Generation .............................................................................130 Recommendation generation ........................................................................131 Information collection phase ...................................................................131 Learning phase .........................................................................................132 Prediction/recommendation phase ..........................................................132 Evaluation metrics for recommendation algorithms ...................................132 Statistical accuracy metric .......................................................................133 Case study on Recommendation system: E-learning system domain .........133 Recommender systems .............................................................................134 Problem definition ....................................................................................134 Objective of the case study .......................................................................134 Considerations for the case study ............................................................135 System development .................................................................................135 Constraints / limitations while developing the recommendation system .....139 Text Analysis and Mining ...................................................................................141 Importance of text mining ............................................................................141 Basic blocks of text mining system using ML ...............................................142 Steps involved in preparing an unstructured text document for deeper analysis...............................................................................................144 Text mining techniques .................................................................................145 Information Retrieval (IR) .......................................................................145 Natural language processing (NLP) .........................................................145 Sentiment analysis .........................................................................................146 Naive Bayes ..............................................................................................147 Linear regression ......................................................................................148 Support Vector Machines (SVM).............................................................148 Deep learning ...........................................................................................150

 xv Case study on product recommendation system based on sentiment analysis .........................................................................................151 Product recommendation ........................................................................151 Problem definition ....................................................................................151 System development .................................................................................152 Considerations for the case study ............................................................154 Opinion mining .............................................................................................157 Image processing ..................................................................................................157 Importance of image processing ....................................................................158 Purpose of image processing..........................................................................158 Basic Image Processing System .....................................................................159 Image Processing using popular ML algorithms ..........................................160 Real Time case studies of Image Processing using ML .................................161 Problem definition ....................................................................................162 Objective of the case study .......................................................................162 Considerations for the case study ............................................................162 System development .................................................................................162 Algorithms that can be employed ............................................................163 Tool Utilization ........................................................................................164 Constraints/Limitations while developing the system .............................164 Evaluation Measures ................................................................................164 Predictive analytics ..............................................................................................165 Importance of predictive analytics ................................................................165 Need for predictive analysis ..........................................................................165 Machine Learning vs Predictive Modeling ...................................................166 Building a predictive model ..........................................................................167 Types of predictive algorithms ......................................................................168 Types of Predictive Analytical Models ..........................................................170 Comparison of Predictive Modeling and Predictive Analytics ....................171 Predictive modeling vs data analytics ...........................................................172 Comparison between and Predictive Analytics and Data Analytics ..........173 Uses or applications of Predictive Analytics .................................................174 Benefits of predictive analytics ......................................................................175 Challenges of predictive modeling ................................................................176 Limitations of predictive modeling ...............................................................177 Case studies ...................................................................................................178

Social media analytics..............................................................................178 Case study of Instagram ...........................................................................181 Case study on Customer churning analytics ...........................................184 Building the hypothesis ............................................................................189 Case study on learning analytics in education systems ..........................192 Challenges faced .......................................................................................193 Approach ..................................................................................................193 Other case studies ................................................................................................196 Conclusion ............................................................................................................196 5. Optimization in ML Algorithm ..................................................................197 Introduction ..........................................................................................................197 Structure ................................................................................................................197 Objectives ..............................................................................................................198 Optimization – Need of ML projects .................................................................198 Types of optimization techniques ..................................................................199 Conventional Approach ...........................................................................199 Metaheuristic Approach ...........................................................................199 Basic Optimization Techniques ....................................................................201 Backpropagation optimization ................................................................202 Gradient descent optimization ................................................................202 Metaheuristic approaches to optimization .......................................................203 Types of metaheuristic algorithms ................................................................205 Single solution-based algorithms .............................................................205 Population-based algorithms ...................................................................207 Improvisation of ML algorithms by optimizing the learning parameters ....214 Case study 1: Metaheuristic Optimization Techniques in Healthcare ........214 Case Study 2: Genetic Algorithm (GA) in Batch Production ......................220 Optimization using Python ................................................................................225 Conclusion ............................................................................................................230 Questions ...............................................................................................................230 Index ..................................................................................................................237

Introduction The term machine learning was coined by Arthur Samuel in 1959. The basic idea behind the coined term was “Can machines do what we as humans can do?” rather than asking “Can machines think?” These questions led to the development of machine learning where, just like human beings, machines tend to learn from experience. The aim is to improve the performance with experience. This chapter introduces the related terms, such as data science, data mining, artificial intelligence, machine learning and deep learning. The major focus is to familiarize you with the methods and techniques used in machine learning. To understand the core concepts, this chapter explores the working process of any given machine learning algorithm. In the recent years, machine learning technology has improved drastically, which is elaborated through the various applications, limitations and the challenges faced while developing the machine learning algorithms. Structure In this chapter, we will discuss the following topics: ● Introduction to Machine Learning Chapter 1 Introduction to ML

2  Python Machine Learning Projects ● Models of Machine Learning o Supervised machine learning model through training o Unsupervised machine learning model o Semi - structured machine learning model o Reinforcement machine learning model ● Working of Machine Learning algorithm ● Challenges for Machine Learning Projects ● Limitations of Machine Learning ● Application areas of Machine Learning ● Difference between the terms data science, data mining, machine learning and deep learning Objectives On completion of this chapter, you will be able to understand the various key terms and the fundamentals of machine learning. Additionally, you will be able to understand the types, the models and the mechanism of machine learning. You will become familiar with the challenges and limitations observed while applying machine learning algorithms. These will be elaborated through a number of application areas along with the differences between the various related technologies employed. Introduction to Machine Learning (ML) Machine learning is defined as the ability of a computer system to learn from the environment where the data is provided through enabling algorithms. These algorithms gather insights and take data-driven decisions with minimal human intervention. This enables us to make predictions on previously unanalyzed data. So instead of writing code, one just needs to feed the data to the generic algorithm, and the algorithm/machine builds the logic based on the given data. This helps improve the output from its experience without the need for any explicit programming. Thus, it can be said that machine learning is a term closely associated with data science, and it involves observing and studying data or experiences to identify patterns and set up a reasoning system based on the findings. Another way of describing machine learning is to say that it is a science of building hardware or software that can achieve tasks by learning from examples. The examples come as {input, output} pairs. When new inputs are given, a trained machine can predict the output. For example, recommendations for online products

Introduction to ML  3 purchased or suggestions for the persons who bought the product is machine learning. The terms used in machine learning include a target that is called a label, a variable used in statistics that is called a feature, and transformation in statistics that is called feature creation. Models in Machine Learning Machine learning models can be categorized into three basic models: supervised, unsupervised and reinforcement learning. Supervised machine learning model through training Supervised learning is said to be a learning obtained by training a machine or a model. On getting trained, predictions can be made for new data, also known as test data. This model works on the data provided to the system. The data available is divided into training and testing data. A supervised learning model analyses the given training data and draws inferences from it. Therefore, mapping between the input and output pair and proper labeling of data is crucial in supervised machine learning models. The major aim of a supervised model is to utilize historical data, understand its behavior and determine future forecasts based on the historical data available and maintained in the database. For example, to differentiate between plants and flowers, a few labeled pictures of both categories need to be fed. This will enable the machine learning algorithm to differentiate between, identify and learn about them based on their characteristics. Once the algorithm is trained, classifying the remaining images would become a lot easier for the algorithm. Unsupervised machine learning model: Self- sufficient learning Such a model does not use any classified or labelled parameters. It learns through observation and finds structures in the data. It focuses on discovering hidden structures, patterns and relationships from unlabeled data, which enhances the functionality of the system by creating a number of clusters for further analysis.

4  Python Machine Learning Projects The unsupervised system or its algorithms are not given the “right answer.” The algorithm is expected to determine, interpret, and review its data and conclude from what is being shown to them through the iterative deep learning approach. Unsupervised learning can use both generative learning models and a retrieval- based approach. These algorithms use self-organizing maps, nearest-neighbor mapping, k-means clustering and singular value decomposition as its techniques for operations. Unsupervised learning also known as neural networks works well for recognizing images, performing operations on transactional data, performing speech to text conversion and natural language generation. Semi-supervised machine learning model This model is a combination of supervised and unsupervised learning. It works using a small amount of labeled and large amount of unlabeled data for training to improve the learning accuracy. Here, all the unlabeled data is fed in, and the machine applies the various algorithms, such as classification, regression and prediction. Then, it understands the characteristics and classifies the information from the data provided. This learning provides an effective solution when the cost associated with labeling is too high. Reinforcement machine learning model: Hit and Trial This learning model interacts with the environment on a trial-and-error basis, determining the best outcome. Reinforcement learning comprises of three primary components: the agent (the learner or decision maker), the environment (everything the agent interacts with) and actions (what the agent can do). The agent here is rewarded or penalized with a point. Based on the actions they take, the output should be maximized over a given amount of time. Steps that produce favorable outcomes are rewarded, and steps that produce undesired outcomes are penalized until the algorithm learns the optimal process. Thus, the goal in reinforcement learning is to learn the best policy to obtain maximum rewards.