Python for Data Science (A. Lakshmi Muddana, Sandhya Vinayakam) (z-library.sk, 1lib.sk, z-lib.sk)

📄 File Format: PDF

💾 File Size: 6.7 MB

📖 Read Online ⬇️ Download

Registered users can read the full content for free

Register as a Gaohf Library member to read the complete e-book online for free and enjoy a better reading experience.

📄 Page 1

A. Lakshmi Muddana Sandhya Vinayakam Python for Data Science

📄 Page 2

Python for Data Science

📄 Page 3

A. Lakshmi Muddana · Sandhya Vinayakam Python for Data Science

📄 Page 4

A. Lakshmi Muddana Department of Computer Science and Engineering Gandhi Institute of Technology and Management (GITAM) Hyderabad, India Sandhya Vinayakam Department of Computer Science and Engineering Gandhi Institute of Technology and Management (GITAM) Hyderabad, India ISBN 978-3-031-52472-1 ISBN 978-3-031-52473-8 (eBook) https://doi.org/10.1007/978-3-031-52473-8 Jointly published with Ane Books Pvt. Ltd. In addition to this printed edition, there is a local printed edition of this work available via Ane Books in South Asia (India, Pakistan, Sri Lanka, Bangladesh, Nepal and Bhutan) and Africa (all countries in the African subcontinent). ISBN of the Co-Publisher’s edition: 978-93-94883-30-7 © The Author(s) 2024 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publishers, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publishers nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publishers remain neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland Paper in this product is recyclable.

📄 Page 5

We dedicate the book to our parents for their inspiration and support Shantha RatnaKumari and Venkateswarlu Mallika and Vinayakam

📄 Page 6

Preface Python is a general-purpose programming language that is used to solve a variety of problems in different domains. It is widely used in Artificial Intelligence, Data Science, Web Development, Internet of Things, etc. Python uses English-like syntax, easy to read and write code. Its vast library support, improved productivity, strong community base, portability, and availability as a free and open source make it very popular among all types of users. The book covers basic and advanced concepts. The basic concepts like its unique features, data types, operators, and developing simple programs using selection and loop statements. As functions are the core of any programming, a detailed illustra- tion of defining and invoking functions, recursive functions, and lambda functions is covered. Built-in data structures of Python are popularly used in data science and model building. Strings, Lists, Tuples, Sets, and Dictionary data structures are discussed in detail with example programming problems. File handling is an important task when handling large data. Data access and manipulation from standard file formats such as CSV, Excel, and JSON files are included in the book. Python is widely used in data analytics and model building. Data manipulations using Pandas and Data visualizations using Matplotlib and Seaborn packages are illustrated with examples and case studies. Regular expressions being an important concept in Natural Language Processing, text manipulation functions are discussed, and a case study is presented with public text data. SQLite3 libraries are discussed for creating and manipulating data in the database. Advanced concepts of building Machine Learning and Deep Learning models and multi-tasking concepts are explained with examples and case studies. The machine learning chapter discusses concepts of supervised and unsupervised learning and model evaluation algorithms using the SciKit-Learn package. The case studies are presented for different learning algorithms using built-in and public datasets. Deep Learning is a sub-field of Machine Learning that mimics the human brain. Concepts of artificial neural networks for both structured and unstructured data are discussed in detail using the Keras library for deep learning. Case studies are presented for stan- dard neural networks, convolutional neural networks, and recurrent neural networks using Keras libraries, vii

📄 Page 7

viii Preface The book also includes a chapter on multi-threading and multi-processing in Python for improving execution time and effective utilization of system resources. Every chapter includes illustrations with examples, worked-out problems, multiple-choice questions for knowledge testing, and exercise problems for prac- tice. Case studies are presented on advanced concepts. The book is supported by a solution manual for multiple-choice questions and exercise problems. The aim and scope of the book are to provide the required knowledge and skill in coding and data analysis. The book serves as the basis for data analysis and model building using Python packages. The reader requires basic maths and logical thinking and no other prerequisites. The book is intended to serve as a textbook for the Problem Solving and Data Science courses of Engineering, Science, and Commerce programs at the undergraduate and postgraduate levels. The key features of the book include: • Basic and advanced programming concepts. • Data manipulation and analysis using Pandas, data visualization, and manipulating text using regular expressions. • File handling and database creation and manipulation concepts. • Machine Learning and Deep Learning models and multi-tasking. • Concepts explained with illustrations and examples. • Case study for an in-depth understanding of concepts on advanced topics using public datasets. • Review questions and exercise problems at the end of chapters. Hyderabad, India A. Lakshmi Muddana Sandhya Vinayakam

📄 Page 8

Acknowledgments We are grateful to Dr. N. Siva Prasad, Retd. Professor IIT Madras, for the motivation in initiating this project. We sincerely acknowledge his constant guidance and support in completing the book. We express our gratitude to Dr. Ghanta Subba Rao, Former Chairman A.P. Skill Development Corporation, for his suggestions and feedback in structuring and writing the book. We sincerely thank our institution authorities, family members, friends, and colleagues for their direct and indirect support. ix

📄 Page 9

Contents 1 Basic Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.3 Data Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.4 Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1.4.1 Arithmetic Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1.4.2 Relational Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.4.3 Logical Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.4.4 Bitwise Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 1.4.5 Assignment Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 1.4.6 Identity Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 1.4.7 Membership Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 1.4.8 Operators Precedence and Associativity . . . . . . . . . . . . . . 16 1.5 Type Conversions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 1.6 Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 1.6.1 Assignment Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 1.6.2 Input Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 1.6.3 Output Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 1.6.4 Conditional Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 1.6.5 Loop Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 1.7 Random Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Review Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 2 Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 2.2 Properties of Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 2.3 Parameters’ Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 2.4 Parameter Passing Mechanisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 2.5 Recursive Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 xi

📄 Page 10

xii Contents 2.6 Lambda Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 Review Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 3 Strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 3.1 Create String Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 3.2 Accessing String Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 3.3 Operations on Strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 3.4 Methods on Strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 3.5 Iterating Through String Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 3.6 Type Conversions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 Review Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 4 Built-in Data Structures: Lists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 4.1 Create List Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 4.2 Indexing and Slicing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 4.3 Nested Lists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 4.4 Methods on List Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 4.5 Functions on List Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 4.6 List Objects in for Loops . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 4.7 List Comprehensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 Review Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 5 Built-in Data Structure: Tuple . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 5.1 Create Tuple Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 5.2 Indexing and Slicing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 5.3 Nested Tuples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 5.4 Operations on Tuples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 5.5 Functions on Tuples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 5.6 Methods on Tuple Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 5.7 Type Conversions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 5.8 Iterating Tuple Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 5.9 Unpacking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 5.10 Enumerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 5.11 Zipping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 Review Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 6 Built-in Data Structure: Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 6.1 Create Set Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 6.2 Properties of Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 6.3 Membership Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 6.4 Functions on Set Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 6.5 Methods on Set Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 6.6 Operations on Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

📄 Page 11

Contents xiii 6.7 Type Conversions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 6.8 Set Comprehensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130 6.9 Frozen Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 Review Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 7 Built-in Data Structure: Dictionary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 7.1 Create Dictionary Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 7.2 Properties of Dictionary Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 7.3 Accessing the Elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 7.4 Operations on Dictionary Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 7.5 Iterating Dictionary Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145 7.6 Dictionary Comprehension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146 7.7 Type Conversions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150 Review Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150 8 Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 8.2 File Handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 8.2.1 Opening the File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 8.2.2 Closing the File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 8.2.3 Writing Data into Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 8.2.4 Get File Pointer Position . . . . . . . . . . . . . . . . . . . . . . . . . . . 158 8.2.5 Reading Data from Files . . . . . . . . . . . . . . . . . . . . . . . . . . . 158 8.2.6 Changing the File Pointer Position . . . . . . . . . . . . . . . . . . 159 8.2.7 Read and Write to Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160 8.3 Working with Files and Directories . . . . . . . . . . . . . . . . . . . . . . . . . 162 8.4 Case Study: File Handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168 Review Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168 9 Data Manipulations with Pandas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171 9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171 9.2 Dataframes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171 9.2.1 Create Dataframes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172 9.2.2 Attributes of a Dataframe . . . . . . . . . . . . . . . . . . . . . . . . . . 173 9.2.3 Add Columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175 9.2.4 Accessing Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176 9.2.5 Adding Rows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 9.2.6 Deleting Columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182 9.2.7 Renaming Column Names and Row Labels . . . . . . . . . . . 183 9.2.8 Methods on Dataframes . . . . . . . . . . . . . . . . . . . . . . . . . . . 184 9.2.9 Functions on Columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185 9.2.10 Operators on Dataframes . . . . . . . . . . . . . . . . . . . . . . . . . . . 186

📄 Page 12

xiv Contents 9.3 Dataframes and Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187 9.4 User-Defined Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190 9.5 Case Study: Data Manipulation and Analysis . . . . . . . . . . . . . . . . . 193 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198 Review Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 10 SQLite3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201 10.1 SQL Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202 10.1.1 Data Definition Language (DDL) . . . . . . . . . . . . . . . . . . . 202 10.1.2 Data Manipulation Language (DML) . . . . . . . . . . . . . . . . 204 10.1.3 Data Query Language (DQL) . . . . . . . . . . . . . . . . . . . . . . . 205 10.1.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205 10.2 Case Study: Database Creation and Operations . . . . . . . . . . . . . . . 210 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214 Review Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215 11 Regular Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217 11.1 Meta Characters and Special Sequences . . . . . . . . . . . . . . . . . . . . . 217 11.2 Functions on Regular Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . 219 11.3 Case Study: Regular Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . 223 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226 Review Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226 12 Data Visualizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229 12.1 Matplotlib . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229 12.2 Seaborn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230 12.3 General Functions in Plotting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230 12.4 Basic Graphs and Plots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231 12.5 Subplots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246 12.6 Case Study: Data Visualizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252 Review Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253 13 Python for Machine Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255 13.1 Data Loading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256 13.2 Data Preparation and Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . 258 13.2.1 Data Cleaning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258 13.2.2 Data Transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263 13.2.3 Splitting the Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267 13.3 Case Study: Preprocessing on the Titanic Dataset . . . . . . . . . . . . . 268 13.4 Supervised Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273 13.4.1 Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273 13.4.2 Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280 13.5 Model Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290 13.5.1 Hyperparameter Tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291 13.5.2 Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293

📄 Page 13

Contents xv 13.6 Ensemble Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295 13.6.1 Basic Ensembling Techniques . . . . . . . . . . . . . . . . . . . . . . 296 13.6.2 Advanced Ensembling Techniques . . . . . . . . . . . . . . . . . . 296 13.6.3 Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300 13.7 Unsupervised Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303 13.7.1 Unsupervised Learning Techniques . . . . . . . . . . . . . . . . . . 303 13.7.2 Clustering Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305 13.7.3 Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309 14 Python for Deep Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321 14.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321 14.2 Data Loading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322 14.2.1 In-built Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322 14.2.2 Loading csv Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323 14.3 Image Data Loading and Preparation . . . . . . . . . . . . . . . . . . . . . . . . 323 14.4 Text Data Loading and Preparation . . . . . . . . . . . . . . . . . . . . . . . . . 328 14.5 Model Building . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335 14.5.1 Activation Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336 14.5.2 Neural Network Layers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336 14.5.3 Methods on the Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340 14.5.4 Model Compilation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340 14.5.5 Model Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341 14.5.6 Model Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341 14.5.7 Model Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341 14.6 Autoencoder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342 14.7 Case Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342 14.7.1 Regression Model on Boston Housing Dataset . . . . . . . . 342 14.7.2 Deep Neural Network for Breast Cancer Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346 14.7.3 Image Classification Model . . . . . . . . . . . . . . . . . . . . . . . . 350 14.7.4 Text Data Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354 15 Python for Multi-tasking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361 15.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361 15.2 Multi-threading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362 15.2.1 Threads Synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . . 366 15.3 Multi-processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369 15.3.1 Interprocess Communication . . . . . . . . . . . . . . . . . . . . . . . 371 15.3.2 Process Pool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377 Review Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377 Appendix A: Solutions to Review Questions . . . . . . . . . . . . . . . . . . . . . . . . . . 379 Appendix B: Python Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389

📄 Page 14

About the Authors A. Lakshmi Muddana received a Ph.D. in Computer Science and Engineering from Osmania University, Hyderabad. She is currently a professor in the Department of Computer Science and Engineering at GITAM Deemed to be University, Hyderabad, India. She has been in academics, teaching undergraduate and postgraduate students and guiding research scholars in the areas of Deep Learning and Security. Sandhya Vinayakam received a Ph.D. in Computer Science and Engineering from Osmania University, Hyderabad. She is currently in the Department of Computer Science and Engineering at GITAM Deemed to be University, Hyderabad, India. She has been in academics and doing research in the areas of Image Processing and Deep Learning. xvii

📄 Page 15

Chapter 1 Basic Python 1.1 Introduction Python is a simple and easy-to-learn high-level programming language suitable for first-time programmers or experienced with other programming languages. Python was created by Guido Van Rossum and released in 1991, which is a successor to the ABC programming language. Its simple syntax makes the program more readable, easy to understand, and debug the code. Python’s popularity is due to its powerful features and applicability in data science and artificial intelligence. It offers basic built-in data structures like lists, dictionaries, sets, tuples, and strings for elegant data organization and manipulation. Unlike other programming languages like C, C++, and Java, Python does not require variables to be declared. It is a dynamically typed language where data types of variables can change dynamically during the program execution. The language has a rich set of standard library functions and community-contributed modules for application development. All these features make Python programs shorter and attractive for rapid application development. Python is an interpreted language that does not require a compilation step. This makes testing, debugging, and prototyping process faster. Being an interpreted lan- guage, programs run slower than C++ and Java but take less time for program devel- opment. Python’s modules and packages encourage modularity and code reusability. The language is extensively used in data science, machine learning, web and API devel- opment, etc. It can be connected to database systems, can read and write into files, and can also handle big data. Python runs on different operating systems and plat- forms like Windows, Mac, Linux, and Raspberry Pi. Python is under an open-source license that makes it freely usable and distributable. Python is a multiparadigm programming language that supports structured, object- oriented, and functional programming. Indentation is based on white spaces to define the scope of statements, loops, functions, and classes, unlike other programming languages like C++ and Java that use curl braces. © The Author(s) 2024 A. L. Muddana and S. Vinayakam, Python for Data Science, https://doi.org/10.1007/978-3-031-52473-8_1 1

📄 Page 16

2 1 Basic Python Sample Python code print("This is Python code") x = 10 print("x is a variable with value ", x) Output This is Python code x is a variable with value 10 Comments in Python Comments in the program provide readability to the code. Python supports the fol- lowing formats for comments: (i) Single line comments: Comments start with # followed by text. It can be spec- ified on a separate line or in the code line. Example # This is a single-line comment print("This is Python code") # code line comment # Following statement assign Value 10 to the variable X x = 10 # print() function display the data print("x is a variable with value ", x) (ii) Multiline comments: Comments can run into multiple lines. Multiline com- ments are enclosed in triple single quotes or triple double quotes. Using triple double quotes Example """ This is Multiline comment in Python """ x = 10 print(x) Output 10

📄 Page 17

1.2 Variables 3 Using triple single quotes Example ”’ This is Multiline comment in Python ”’ x = 10 print(x) Output 10 1.2 Variables Variable is the name of the memory location that can store a value. The value can be numeric or text or Boolean type. Example # variable x x = 10 With the above code, the value 10 is stored in the variable named x. How to Name a Variable? A variable name is a sequence of characters with the following rules: . • Name can contain alphabets, digits, and underscore (. _). . • The first character cannot be a digit. . • Name cannot be a Python keyword. . • Name is case sensitive (variable name Total is different from variable total). . • By convention, variable names are in all lowercase with underscore separating words. For example, max. _value. Python Keywords are reserved words used for specific purposes and cannot be used for general purposes by the user. The following table shows the keywords used in Python (Table 1.1).

📄 Page 18

4 1 Basic Python Table 1.1 Python keywords and as assert async await break class continue def del else elif except False finally for from global if import in is lambda None nonlocal not or pass raise return True try while with yield Variable names are case sensitive. Example count=10 # count is variable with value 10 Count=20 # Count is variable with value 20 print(count,Count) Output In the above example, count and Count are two different variables and allocated memory separately as shown in the above figure. 1.3 Data Types A variable can hold different types of data. Python supports the following basic data types (Fig. 1.1). (i) Numerical data types can hold integer value represented as int or real value represented as float or complex values represented as complex. (ii) Boolean data type is represented as bool. (iii) Text data type is represented as str(string). Python Feature 1: Variables are Not Declared Unlike other high-level programming languages, Python variables are not declared. The data type of a variable is determined by the value it holds. The data type of the variable can be determined using the type() function.

📄 Page 19

1.3 Data Types 5 Fig. 1.1 Data types Example x = 10.5 # Here x is a float variable as it is having decimal value print(x) # display x value print(type(x)) # display data type of x Output 10.5 . <class ’float’. > Python Feature 2: Python is an object-oriented language. It implements data types as classes like class int, class float, class bool, etc. (i) Integer data as class int Example i = 10 # i is a integer variable print(type(i)) # display data type of i Output . <class ’int’. > (ii) Real values as class float Example pi = 3.14 # pi is a float variable print(type(pi)) # display data type of pi Output . <class ’float’. > (iii) Complex values as class complex. Python uses j to indicate the imaginary part.

📄 Page 20

6 1 Basic Python Example c = 10 + 5j # Here 10 is real part & 5 imaginary part print(type(c)) # Display data type of the variable c print(c) # Display value of c Output . <class ’complex’. > (10+5j) (iv) Boolean data as class bool. Boolean variables can have either True or False values. Example b1 = True b2 = False print(type(b1), b1) print(type(b2), b2) Output . <class ’bool’.> True . <class ’bool’.> False (v) Text data as class str. A string is a sequence of characters enclosed in single or double quotes. Example s1 = "Hello everyone" # text enclosed in double quotes s2 = ’Good morning’ # text enclosed in single quotes print(type(s1), type(s2)) print(s1) print(s2) Output . <class ’str’.> . <class ’str’. > Hello everyone Good morning Python supports user-defined classes. Instances of that class can be created called as objects. Example # user defined class class circle: def __init__(self, r): self.radius = r