Machine Learning and AI with Simple Python and Matlab Scripts. Courseware for Non-computing Majors (M. Ümit Uyar) (z-library.sk, 1lib.sk, z-lib.sk)

📄 File Format: PDF

💾 File Size: 6.8 MB

📖 Read Online ⬇️ Download

Registered users can read the full content for free

Register as a Gaohf Library member to read the complete e-book online for free and enjoy a better reading experience.

📄 Page 1

(This page has no text content)

📄 Page 2

(This page has no text content)

📄 Page 3

Machine Learning and AI with Simple Python and Matlab Scripts

📄 Page 4

IEEE Press 445 Hoes Lane Piscataway, NJ 08854 IEEE Press Editorial Board Sarah Spurgeon, Editor-in-Chief Moeness Amin Jón Atli Benediktsson Adam Drobot James Duncan Ekram Hossain Brian Johnson Hai Li James Lyke Joydeep Mitra Desineni Subbaram Naidu Tony Q. S. Quek Behzad Razavi Thomas Robertazzi Patrick Chik Yue

📄 Page 5

Machine Learning and AI with Simple Python and Matlab Scripts Courseware for Non-computing Majors M. Ümit Uyar The City College of New York New York, USA

📄 Page 6

Copyright © 2025 by The Institute of Electrical and Electronics Engineers, Inc. All rights reserved. Published by John Wiley & Sons, Inc., Hoboken, New Jersey. Published simultaneously in Canada. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 750-4470, or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley.com/ go/permission. The manufacturer’s authorized representative according to the EU General Product Safety Regulation is Wiley-VCH GmbH, Boschstr. 12, 69469 Weinheim, Germany, e-mail: Product_Safety@wiley.com. Trademarks: Wiley and the Wiley logo are trademarks or registered trademarks of John Wiley & Sons, Inc. and/or its affiliates in the United States and other countries and may not be used without written permission. All other trademarks are the property of their respective owners. John Wiley & Sons, Inc. is not associated with any product or vendor mentioned in this book. Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Further, readers should be aware that websites listed in this work may have changed or disappeared between when this work was written and when it is read. Neither the publisher nor authors shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages. For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at (800) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002. Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic formats. For more information about Wiley products, visit our web site at www.wiley.com. Library of Congress Cataloging-in-Publication Data Applied for Hardback ISBN: 9781394294954 Cover Design: Wiley Cover image is a painting titled “Yeni Arkadaş” (New Friend) by Mümtaz Yener (private collection of his daugther Prof. Dr. Göksun Nina Say-Yener) Author Photo: Courtesy of M. Ümit Uyar Set in 9.5/12.5pt STIXTwoText by Straive, Chennai, India

📄 Page 7

To my beloved daughters Aylin Emine and Melisa Ayşe

📄 Page 8

(This page has no text content)

📄 Page 9

vii Contents About the Author xiii Preface xv Acknowledgments xvii About the Companion Website xix 1 Introduction 1 1.1 Artificial Intelligence 1 1.2 A Historical Perspective 1 1.3 Principles of AI 2 1.4 Applications That Are Impossible Without AI 2 1.5 Organization of This Book 3 2 Artificial Neural Networks 7 2.1 Introduction 7 2.2 Applications of ANNs 7 2.3 Components of ANNs 8 2.3.1 Neurons 8 2.3.2 Sigmoid Activation Function 9 2.3.3 Rectilinear Activation Function 9 2.3.4 Weights of Synapses 10 2.4 Training an ANN 11 2.5 Forward Propagation 12 2.5.1 Forward Propagation from Input to Hidden Layer 13 2.6 Back Propagation 13 2.6.1 Back Propagation for a Neuron 13 2.6.2 Back Propagation – from Output to Hidden Layer 15 2.6.3 Back Propagation – from Hidden Layer to Input 16 2.7 Updating Weights 17 2.8 ANN with Input Bias 17 2.9 A Simple Algorithm for ANN Training 18 2.10 Computational Complexity of ANN Training 18 2.11 Normalization of ANN Inputs and Outputs 19 2.12 Concluding Remarks 20 2.13 Exercises for Chapter 2 20

📄 Page 10

viii Contents 3 ANNs for Optimized Prediction 23 3.1 Introduction 23 3.2 Selection of ANN Inputs 24 3.3 Selection of ANN Outputs 24 3.4 Construction of Hidden Layers 25 3.5 Case Study 1: Sleep-Study Example 25 3.5.1 Using Matrices for ANN Training 26 3.5.2 Forward Propagation 28 3.5.3 Back Propagation 28 3.5.4 Updating Weights 29 3.5.5 Forward Propagation with New Weights 29 3.5.6 Back Propagation with New Weights 30 3.5.7 Using Normalized Input and Output Values 31 3.5.8 Reducing Errors During Training 34 3.5.9 Implementation of Sleep-Study ANN in Python 34 3.5.10 Implementation of Sleep-Study ANN in Matlab 37 3.6 Case Study 2: Prediction of Bike Rentals 41 3.6.1 Python Script for Bike Rentals Using an ANN 41 3.6.2 Matlab Script for Bike Rentals Using an ANN 46 3.7 Concluding Remarks 48 3.8 Exercises for Chapter 3 48 4 ANNs for Financial Stock Trading 51 4.1 Introduction 51 4.2 Programs that Buy and Sell Stocks 51 4.3 Technical Indicators 51 4.3.1 Simple Moving Average 52 4.3.2 Momentum 53 4.3.3 Exponential Moving Average 54 4.3.4 Bollinger Bands 54 4.4 A Simple Algorithmic Trading Policy 55 4.5 A Simple ANN for Algorithmic Stock Trading 57 4.5.1 ANN Inputs and Outputs 57 4.5.2 ANN Architecture 58 4.6 Python Script for Stock Trading Using an ANN 59 4.7 Matlab Script for Stock Trading Using an ANN 63 4.8 Concluding Remarks 65 4.9 Exercises for Chapter 4 65 5 ANNs for Alzheimer’s Disease Prognosis 67 5.1 Introduction 67 5.2 Alzheimer’s Disease 67 5.3 A Simple ANN for AD Prognosis 68 5.4 Python Script for AD Prognosis Using an ANN 71 5.5 Matlab Script for AD Prognosis Using an ANN 75 5.6 Concluding Remarks 80 5.7 Exercises for Chapter 5 81

📄 Page 11

Contents ix 6 ANNs for Natural Language Processing 83 6.1 Introduction 83 6.2 Impact of Text Messages on Stock Markets 84 6.3 A Simple ANN for NLP 85 6.3.1 ANN Inputs and Outputs 85 6.3.2 Keywords 85 6.3.3 Formation of Training Data 86 6.3.4 ANN Architecture 88 6.4 Python Script for NLP Using an ANN 89 6.5 Matlab Script for NLP Using an ANN 92 6.6 Concluding Remarks 96 6.7 Exercises for Chapter 6 97 7 Convolutional Neural Networks 99 7.1 Introduction 99 7.1.1 Training CNNs 100 7.2 Variations of CNNs 101 7.3 Applications of CNNs 101 7.4 CNN Components 102 7.5 A Numerical Example of a CNN 102 7.6 Computational Cost of CNN Training 108 7.7 Concluding Remarks 112 7.8 Exercises for Chapter 7 112 8 CNNs for Optical Character Recognition 115 8.1 Introduction 115 8.2 A Simple CNN for OCR 115 8.3 Organization of Training and Reference Files 117 8.4 Python Script for OCR Using a CNN 119 8.5 Matlab Script for OCR Using a CNN 124 8.6 Concluding Remarks 130 8.7 Exercises for Chapter 8 130 9 CNNs for Speech Recognition 133 9.1 Introduction 133 9.2 A Simple CNN for Speech Recognition 134 9.3 Organization of Training and Reference Files 136 9.4 Python Script for Speech Recognition Using a CNN 138 9.5 Matlab Script for Speech Recognition Using a CNN 144 9.6 Concluding Remarks 150 9.7 Exercises for Chapter 9 150 10 Recurrent Neural Networks 151 10.1 Introduction 151 10.2 One-to-One Single RNN Cell 153 10.2.1 A Simple Alphabet and One-Hot Encoding 156 10.2.2 Forward and Back Propagation 157

📄 Page 12

x Contents 10.3 A Numerical Example 158 10.4 Multiple Hidden Layers 163 10.5 Embedding Layer 165 10.5.1 Forward and Back Propagation with Embedding 167 10.5.2 A Numerical Example with Embedding 168 10.6 Concluding Remarks 172 10.7 Exercises for Chapter 10 172 11 RNNs for Chatbot Implementation 175 11.1 Introduction 175 11.2 Many-to-Many RNN Architecture 175 11.3 A Simple Chatbot 176 11.4 Python Script for a Chatbot Using an RNN 179 11.5 Matlab Script for a Chatbot Using an RNN 183 11.6 Concluding Remarks 188 11.7 Exercises for Chapter 11 189 12 RNNs with Attention 191 12.1 Introduction 191 12.2 One-to-One RNN Cell with Attention 191 12.3 Forward and Back Propagation 193 12.4 A Numerical Example 195 12.5 Embedding Layer 200 12.6 A Numerical Example with Embedding 202 12.7 Concluding Remarks 207 12.8 Exercises for Chapter 12 207 13 RNNs with Attention for Machine Translation 209 13.1 Introduction 209 13.2 Many-to-Many Architecture 210 13.3 Python Script for Machine Translation by an RNN-Att 211 13.4 Matlab Script for Machine Translation by an RNN-Att 216 13.5 Concluding Remarks 223 13.6 Exercises for Chapter 13 223 14 Genetic Algorithms 225 14.1 Introduction 225 14.2 Genetic Algorithm Elements 226 14.3 A Simple Algorithm for a GA 227 14.4 An Example of a GA 230 14.5 Convergence in GAs 231 14.6 Concluding Remarks 232 14.7 Exercises for Chapter 14 232 15 GAs for Dietary Menu Selection 235 15.1 Introduction 235 15.2 Definition of the KP 236

📄 Page 13

Contents xi 15.3 A Simple Algorithm for the KP 238 15.4 Variations of the KP 239 15.5 GAs for KP Solution 240 15.6 Python Script for Dietary Menu Selection Using a GA 242 15.7 Matlab Script for Dietary Menu Selection Using a GA 245 15.8 Concluding Remarks 248 15.9 Exercises for Chapter 15 248 16 GAs for Drone Flight Control 251 16.1 Introduction 251 16.2 UAV Swarms 251 16.3 UAV Flight Control 252 16.4 A Simple GA for UAV Flight Control 253 16.4.1 Virtual Force-Based Fitness Function 254 16.4.2 FGA Progression 255 16.4.3 Chromosome for FGA 257 16.5 Python Script for UAV Flight Control Using a GA 260 16.6 Matlab Script for UAV Flight Control Using a GA 264 16.7 Concluding Remarks 270 16.8 Exercises for Chapter 16 271 17 GAs for Route Optimization 273 17.1 Introduction 273 17.2 Definition of the TSP 274 17.3 A Simple Algorithm for the TSP 276 17.4 Variations of the TSP 277 17.5 GA Solution for the TSP 277 17.6 Python Script for Route Optimization Using a GA 279 17.7 Matlab Script for Route Optimization Using a GA 284 17.8 Concluding Remarks 287 17.9 Exercises for Chapter 17 289 18 Evolutionary Methods 291 18.1 Introduction 291 18.2 Particle Swarm Optimization 291 18.2.1 Applications of PSO 292 18.2.2 PSO Operation 293 18.2.3 Remarks for PSO 298 18.3 Differential Evolution 298 18.3.1 Different Versions of DE 299 18.3.2 Applications of DE 299 18.3.3 A Simple Algorithm for DE 299 18.3.4 Numerical Example: Maximum of sinc by DE 302 18.3.5 Remarks for DE 305 18.4 Grammatical Evolution 306 18.4.1 A Simple Algorithm for GE 306 18.4.2 Definition of GE 307

📄 Page 14

xii Contents 18.4.3 A Simple GA to Implement GE 314 18.4.4 Remarks on GE 315 Appendix A ANNs with Bias 317 A.1 Introduction 317 A.2 Training with Bias Input 317 A.3 Forward Propagation 318 A.3.1 Forward Propagation from Input to Hidden Layer 319 A.3.2 Neuron Back Propagation with Bias Input 319 Appendix B Sleep Study ANN with Bias 321 B.1 Inclusion of Bias Term in ANN 321 B.1.1 Inclusion of Bias in Matrices 321 B.1.2 Forward Propagation with Biases 322 Appendix C Back Propagation in a CNN 327 Appendix D Back Propagation Through Time in an RNN 331 D.1 Back Propagation in an RNN 331 D.2 Embedding Layer 335 Appendix E Back Propagation Through Time in an RNN with Attention 337 E.1 Back Propagation in an RNN-Att 337 E.2 Embedding Layer 340 Bibliography 343 Index 353

📄 Page 15

xiii About the Author Dr. M. Ümit Uyar is a professor at the City College of the City University of New York. He is the co-founder and current director of the Computer Engineering program at the City College. He was the lead and co-principal investigator for large grants to conduct research on AI and game theory-based autonomous software agents, knowledge sharing mobile agents using bio-inspired algorithms for topology control in mobile networks, tactical interoperability of combat networks and efficient reliable end-to-end communications. Based on his research experience in civilian and military telecom- munication protocols and mobile ad hoc computer networks, Dr. Uyar developed AI and game theory algorithms for large swarms of autonomous drones and intelligent techniques for personalized prognosis of degenerative diseases such as Alzheimer’s disease and cancer. While in industry, he developed widely used software tools that improved the software development cycle by orders of magnitude. He has published extensively in AI, game theory and formal description techniques used for complex telecommunication protocols, edited books and co-chaired international conferences. Prior to joining academia, he was a Distinguished Member of Technical Staff at AT&T Bell Labs. He is an IEEE Fellow and holds seven US patents. Dr. Uyar has a BS degree from İstanbul Teknik Üniversitesi, and MS and PhD degrees from Cornell University, all in electrical engineering.

📄 Page 16

(This page has no text content)

📄 Page 17

xv Preface This book is an introduction to artificial intelligence (AI)-based problem solving techniques, together with the basic concepts they originated from and their computational principles. Readers will see how several real-life problems can be modelled so that AI methods can be applied to solve them. Each chapter focuses on a realistic challenge but with reduced dimensions, presenting a sim- ple, easy-to-follow solution as both Python and Matlab scripts. Projects include making financial stock market predictions based on text messages from famous people, algorithmic trading of finan- cial assets, bike rental predictions for the City of London, personalized prediction of Alzheimer’s disease based on genetic information, speech recognition, chatbot implementation and translation of written text from one language to another. The emphasis on these projects is not to write programs to implement the AI algorithms in computer languages but rather to demonstrate the relative simplicity of the AI algorithms used to solve these problems. After finishing this book, it will become apparent to the reader that although much has been accomplished in the world of AI, there is still much more to be learned. This book will prepare them for tackling a wide range of complex problems by applying cutting-edge AI techniques and advancing on new frontiers. About the cover: Yeni Arkadaş (New Friend) by Mümtaz Yener (private collection of his daughter Prof. Dr. Nina Göksun Say-Yener). Mümtaz Yener (1918–2007) worked and lived in Istanbul as a painter with his best friend and wife Şadan Yener. His extensive artwork has been critically acclaimed around the world and featured in renowned museum exhibitions and private art collections in Turkey, Europe, the United States, Japan and Brazil. He believed that “freedom depends on the amount of domination that humans have over machines.”

📄 Page 18

(This page has no text content)

📄 Page 19

xvii Acknowledgments I am grateful for the comments that I received about the materials presented in this book from the undergraduate and graduate students who have taken my classes at the City University of New York over the years. Their relentless pursuit of knowledge has always been refreshing. Specifically, I acknowledge the efforts of the following past and present students of the City College of New York: ● Olga Chsherbakova for speedy implementation of Python and Matlab scripts ● Hasan Şuca Kayman for his skill in deriving complex gradients and implementing scripts for recurrent neural network and attention models ● Dr. Samrat S. Batth for contributions to the chapter on financial applications ● Michal Kropiewnicki for critical reading and constructive comments ● Kelvin Ma for his contributions to the manuscript and projects ● Clement McClean for preparing student assignments with admirable ease ● Grace McGrath, Joe Malubay, Jian Wen Choong, Ishmam Fardin and Sultana Begum for contri- butions to the Alzheimer’s disease prognosis scripts ● Ricardo Valdez for his contributions to the manuscript and projects and for his derivation of ANN examples with an impressive show of patience ● Alexander Verzun and Roberto Behar for corrections to the text and scripts I thank my friend and colleague Prof. Janusz Kusyk for his invaluable contributions on all concepts in this book throughout the chapters. I thank Prof. Dr. Nina Göksun Say-Yener for generously sharing Mümtaz Yener’s masterpiece Yeni Arkadaş (New Friend) as the cover of this book. I thank the Wiley team for their support and understanding: Mary Hatcher, Brett Kurzman, Ryan Coach, Dimple Philip, Vijayalakshmi Saminathan and Akhil Ajikumar. Special thanks go to Chris Cartwright for his meticulous editing. Lastly, I would like to thank Aylin E. Uyar for simplifying complex concepts covered through- out and making them more accessible to a wider audience, including readers without a software engineering background. August 2024 M. Ümit Uyar, PhD The City College of New York, New York, USA

📄 Page 20

(This page has no text content)