Practical Artificial Intelligence with Swift From Fundamental Theory to Development of AI-Driven Apps (Mars Geldard, Jonathon Manning etc.) (z-library.sk, 1lib.sk, z-lib.sk)

📄 File Format: PDF

💾 File Size: 43.4 MB

📖 Read Online ⬇️ Download

Registered users can read the full content for free

Register as a Gaohf Library member to read the complete e-book online for free and enjoy a better reading experience.

📄 Page 1

Mars Geldard, Jonathon Manning, Paris Buttfield-Addison & Tim Nugent Practical Artificial Intelligence with Swift From Fundamental Theory to Development of AI-Driven Apps

📄 Page 2

(This page has no text content)

📄 Page 3

Mars Geldard, Jonathon Manning, Paris Buttfield-Addison, and Tim Nugent Practical Artificial Intelligence with Swift From Fundamental Theory to Development of AI-Driven Apps Boston Farnham Sebastopol TokyoBeijing

📄 Page 4

978-1-492-04481-9 [LSI] Practical Artificial Intelligence with Swift by Mars Geldard, Jonathon Manning, Paris Buttfield-Addison, and Tim Nugent Copyright © 2020 Secret Lab. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com . Acquisition Editor: Rachel Roumeliotis Developmental Editor: Michele Cronin Production Editor: Nan Barber Copyeditor: Kim Cofer Proofreader: Octal Publishing, LLC Indexer: WordCo Indexing Services, Inc. Interior Designer: David Futato Cover Designer: Karen Montgomery Illustrator: Rebecca Demarest October 2019: First Edition Revision History for the First Edition 2019-10-18: First Release See http://oreilly.com/catalog/errata.csp?isbn=9781492044819 for release details. The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Practical Artificial Intelligence with Swift, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc. The views expressed in this work are those of the authors, and do not represent the publisher’s views. While the publisher and the authors have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the authors disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights.

📄 Page 5

Praise for Practical Artificial Intelligence with Swift In the long run, the vast majority of AI will be consumed not in data centers, but where the data actually are—on billions of real-world devices. With its combination of speed and expressiveness, Swift has emerged as the language of choice to enable this transition. This book provides a fantastic introduction to the tools forming the foundation of the next decade of AI. —Dr. Jameson Toole, Cofounder and CTO of Fritz AI The importance of bananas and apples to machine learning developers cannot be overstated. Fortunately this book has a sufficiency of both, or at least, enough to be getting on with. So if, like me, you’re starting to suspect that the future of machine learning is on device, this book is a good place to get started. —Dr. Alasdair Allan, Babilim Light Industries Over the past several years, AI has transitioned from academia and science fiction to a pragmatic technology for solving real-world problems. This book shows how to bring AI right to the palm of your hand, implementing ML capabilities that are only now becoming possible. I’m incredibly excited to see what readers do with what they learn from this book. —Chris Devers, Technical Lead of Sustaining Engineering, EditShare

📄 Page 6

(This page has no text content)

📄 Page 7

Table of Contents Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi Part I. Fundamentals and Tools 1. Artificial Intelligence!?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Practical AI with Swift…and Python? 2 Code Examples 3 Why Swift? 5 Why AI? 7 What Is AI and What Can It Do? 9 Deep Learning versus AI? 11 Where Do the Neural Networks Come In? 11 Ethical, Effective, and Appropriate Use of AI 11 Practical AI Tasks 16 A Typical Task-Based Approach 16 2. Tools for Artificial Intelligence. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 Why Top Down? 19 Great Tools for Great AI 20 Tools from Apple 22 CoreML 22 CreateML 31 Turi Create 37 Apple’s Other Frameworks 40 CoreML Community Tools 42 Tools from Others 44 Swift for TensorFlow 44 v

📄 Page 8

TensorFlow to CoreML Model Converter 45 Other Converters 46 AI-Adjacent Tools 46 Python 46 Keras, Pandas, Jupyter, Colaboratory, Docker, Oh My! 52 Other People’s Tools 55 What’s Next? 55 3. Finding or Building a Dataset. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 Planning and Identifying Data to Target 57 Negation as Failure 58 Closed-World Assumptions 59 Finding a Dataset 60 Where to Look 60 What to Look Out for 62 Building a Dataset 63 Data Recording 63 Data Collation 64 Data Scraping 64 Preparing a Dataset 65 Getting to Know a Dataset 65 Cleaning a Dataset 67 Transforming a Dataset 68 Verifying the Suitability of a Dataset 70 Apple’s Models 70 Part II. Tasks 4. Vision. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 Practical AI and Vision 73 Task: Face Detection 74 Problem and Approach 75 Building the App 75 What Just Happened? How Does This Work? 86 Improving the App 86 Even More Improvements 89 Task: Barcode Detection 98 Task: Saliency Detection 105 Task: Image Similarity 109 Problem and Approach 109 Building the App 110 vi | Table of Contents

📄 Page 9

What Just Happened? How Does This Work? 119 Next Steps 119 Task: Image Classification 120 Problem and Approach 120 Building the App 123 AI Toolkit and Dataset 130 Incorporating the Model in the App 137 Improving the App 140 Task: Drawing Recognition 142 Problem and Approach 142 AI Toolkit and Dataset 144 Building the App 152 What’s Next? 163 Task: Style Classification 165 Converting the Model 165 Using the Model 168 Next Steps 170 5. Audio. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173 Audio and Practical AI 173 Task: Speech Recognition 173 Problem and Approach 174 Building the App 175 What Just Happened? How Does This Work? 182 What’s Next? 182 Task: Sound Classification 184 Problem and Approach 184 Building the App 186 AI Toolkit and Dataset 196 Creating a Model 200 Incorporating the Model in the App 202 Improving the App 207 Next Steps 211 6. Text and Language. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213 Practical AI, Text, and Language 213 Task: Language Identification 214 Task: Named Entity Recognition 217 Task: Lemmatization, Tagging, and Tokenization 218 Parts of Speech 221 Tokenizing a Sentence 222 Task: Sentiment Analysis 225 Table of Contents | vii

📄 Page 10

Problem and Approach 225 Building the App 226 AI Toolkit and Dataset 232 Creating a Model 233 Incorporating the Model in the App 237 Task: Custom Text Classifiers 241 AI Toolkit and Dataset 242 Next Steps 249 7. Motion and Gestures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255 Practical AI, Motion, and Gestures 255 Task: Activity Recognition 255 Problem and Approach 256 Building the App 256 What Just Happened? How Does This Work? 261 Task: Gestural Classification for Drawing 262 Problem and Approach 263 AI Toolkit and Dataset 264 Building the App 265 Task: Activity Classification 277 Problem and Approach 278 AI Toolkit and Dataset 278 Using the Model 286 Task: Using Augmented Reality with AI 287 Next Steps 288 8. Augmentation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289 Practical AI and Augmentation 289 Task: Image Style Transfer 290 Problem and Approach 290 Building the App 292 AI Toolkit and Dataset 307 Creating a Model 308 Incorporating the Model in the App 309 Task: Sentence Generation 315 What Just Happened? How Does This Work? 321 Task: Image Generation with a GAN 324 Problem and Approach 324 AI Toolkit and Dataset 324 Building an App 335 Task: Recommending Movies 344 Problem and Approach 344 viii | Table of Contents

📄 Page 11

AI Toolkit and Dataset 344 Using a Recommender 354 Task: Regressor Prediction 357 Problem and Approach 357 AI Toolkit and Dataset 358 Using the Regressor in an App 364 Next Steps 365 9. Beyond Features. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367 Task: Installing Swift for TensorFlow 368 Adding Swift for TensorFlow to Xcode 368 Installing Swift for TensorFlow with Docker and Jupyter 371 Using Python with Swift 378 Task: Training a Classifier Using Swift for TensorFlow 381 Task: Using the CoreML Community Tools 382 The Problem 382 The Process 382 Using the Converted Model 386 On-Device Model Updates 389 Task: Downloading Models On-device 390 Next Steps 391 Part III. Beyond 10. AI and ML Methods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395 Terminology 396 AI/ML Components 396 AI/ML Objectives 396 Types of Values 397 Classification 398 Methods 399 Applications 426 Clustering 430 Methods 430 Applications 435 Next Steps 437 11. Looking Under the Hood. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439 A Look Inside CoreML 439 Vision 446 Face Detection 447 Table of Contents | ix

📄 Page 12

Barcode Detection 448 Saliency Detection 449 Image Classification 449 Image Similarity 451 Bitmap Drawing Classification 453 Audio 454 Sound Classification 454 Speech Recognition 456 Text and Language 459 Language Identification 460 Named Entity Recognition 461 Lemmatization, Tagging, Tokenization 464 Recommendations 465 Prediction 465 Text Generation 466 Generation 468 The Future of CoreML 470 Next Steps 472 12. The Hard Way. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 473 Behind CoreML’s Magic 473 Task: Building XOR 476 The Shape of Our Network 477 The Code 478 Building It Up 479 Making It Work 484 Tearing It Down 485 Using the Neural Network 486 Approximations of XOR 486 Training 487 Next Steps 490 Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 493 x | Table of Contents

📄 Page 13

Preface Welcome to Practical Artificial Intelligence with Swift. This book takes a task-based approach to practical implementations of artificial intel‐ ligence (AI) using Swift. We do this because we think that you shouldn’t need to dive deep into complicated mathematics and algorithms in order to have clever AI- and machine learning-driven features in your iOS application. AI shouldn’t be a specialist area that’s available only to AI experts. AI should be accessible. We live in a world where these techniques are becoming more than commonplace. They’re becoming part of the fabric of how we interact with computers. Because of the incredible power of machine learning, and its ability to impact, benefit, influence, and control human beings, it’s as important for people to be able to have knowledge over how to build and understand artificial intelligence as it was to know how to operate a computer. To that end, this book is designed to give you a practical understanding of common machine learning tasks. With these, you’ll be able to build better tools, and under‐ stand the behaviour of the tools used by others in the wider world. We’re glad you’re here. Let’s build the machine. Resources Used in This Book We recommend following along with the book by writing code yourself as you pro‐ gress through each chapter. If you become stuck, or just want to archive a copy of the code, you can find what you need via our website. Audience and Approach This book is written for people who already know how to program in Swift, and want to learn about the specific features and techniques that power machine learning. This xi

📄 Page 14

isn’t a book that will teach you the basics of programming, which means that the examples will expect a reasonable familiarity with the language. In addition to using Swift, we’ll occasionally dip into Python from time to time. Python is incredibly common in the world of machine learning and artificial intelli‐ gence. The meat of the book is in Swift, though, and we explain why whenever we’re using Python. We’re going to remind you about this a few times. Finally, we also assume that you’re fairly comfortable navigating macOS and iOS as a user, and that you have access to a Mac to develop on, as well as an iOS device to test on, for the tasks that require certain sensors, like motion tracking or analysing the camera. Organization of This Book This book is divided into three parts. In Part I, Fundamentals and Tools, we introduce the fundamental ideas behind machine learning and artificial intelligence. We set you up with the languages and tools you’ll be using, and get you ready to build useful things. In Part II, Tasks, we visit a wide range of interesting topics, ranging from computer vision, audio, motion, and language. For each topic, we’ll present and build an app that highlights various techniques and APIs; at the end of each chapter, you’ll have a complete working demo app that shows off a practical task that your code can now accomplish. In Part III, Beyond, we take a look behind the curtain, and examine the techniques that power the apps in Part II in detail. We’ll look closely at the theory of machine learning, and provide a deeper understanding of what it is that your device is actually doing. Using This Book We want this book to be deeply founded in the practical. Because of this, we’ve struc‐ tured it around the practical tasks that you might want to do when dealing with artifi‐ cial intelligence and Swift. We’ve split the book into three parts. Part I discusses Swift and AI and the task-based approach to AI that we’re taking for the book (you’re reading a chapter in that part right now), the tools you can use for Swift and AI (that’s Chapter 2), and how and why to approach the datasets you use for your practical artificial intelligences (that’s the third chapter in this part). Part II, chapter by chapter, explores the different domains for which AI can be inte‐ grated into your Swift apps from a thematic point of view: vision-related tasks (Chap‐ ter 4), audio tasks (Chapter 5), language and text tasks (Chapter 6), motion and xii | Preface

📄 Page 15

gestural tasks (Chapter 7), augmentation tasks (Chapter 8) for generating and recom‐ mending things, and beyond (Chapter 9) to explore the more advanced tools and frameworks we touched on at the end of the first part. We outline what each of the tasks entails in the next section. Part III explores how the AI methods actually work (Chapter 10), how the practical tasks we made in Part II work under the hood (Chapter 11), and finally how you can implement simple versions of these things yourself, the hard way (Chapter 12). Our Tasks In this book, we cover the following tasks in the following domains: • Vision — Face detection — Barcode detection — Saliency detection — Image similarity — Image classification — Drawing recognition — Style classification • Audio — Speech recognition — Sound classification • Text and Language — Language identification — Named entity recognition — Lemmatization, tagging, tokenization — Sentiment analysis — Custom text classifiers • Motion and gestures — Activity recognition — Gesture classification for drawings — Activity classification — Using augmented reality with AI Preface | xiii

📄 Page 16

• Augmentation — Image style transfer — Sentence generation — Image generation — Movie recommendation — Regression • Beyond — Installing Swift for TensorFlow — Using Python with Swift — Training a classifier using Swift for TensorFlow — Using the CoreML Community Tools — On-device model updates — Downloading models on device The bulk of this book is dedicated to exploring how you can implement AI-driven things in your apps, (mostly) using Swift. Because we take a top-down, task-focused perspective to AI, we decided to front load our discussion of the tools that you might encounter. We do this in Chapter 2. There, we explore some of the tools that you can use to create models for machine learning and AI tasks with Swift as well as some of the tools for manipulating models, working with data, and generally doing practical AI things with Swift. Conventions Used in This Book The following typographical conventions are used in this book: Italic Indicates new terms, URLs, email addresses, filenames, and file extensions. Constant width Used for program listings, as well as within paragraphs to refer to program ele‐ ments such as variable or function names, databases, data types, environment variables, statements, and keywords. Also used for commands and command- line output. xiv | Preface

📄 Page 17

This element signifies a tip or suggestion. This element signifies a general note. This element indicates a warning or caution. Using Code Examples Supplemental material (code examples, exercises, errata, etc.) is available for down‐ load at our website. This book is here to help you get your job done. In general, if example code is offered with this book, you may use it in your programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require per‐ mission. We appreciate, but do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: “Practical Artificial Intelligence with Swift, 1st Edition, by Mars Geldard, Jonathon Manning, Paris Buttfield-Addison, and Tim Nugent. Copyright 2020 Secret Lab, 978-1-492-04481-9. If you feel your use of code examples falls outside fair use or the permission given above, feel free to contact us at permissions@oreilly.com. Preface | xv

📄 Page 18

O’Reilly Online Learning For over 40 years, O’Reilly Media has provided technology and business training, knowledge, and insight to help companies succeed. Our unique network of experts and innovators share their knowledge and expertise through books, articles, conferences, and our online learning platform. O’Reilly’s online learning platform gives you on-demand access to live training courses, in- depth learning paths, interactive coding environments, and a vast collection of text and video from O’Reilly and 200+ other publishers. For more information, please visit http://oreilly.com. How to Contact Us Please address comments and questions concerning this book to the publisher: O’Reilly Media, Inc. 1005 Gravenstein Highway North Sebastopol, CA 95472 800-998-9938 (in the United States or Canada) 707-829-0515 (international or local) 707-829-0104 (fax) We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at https://oreil.ly/practical-ai-swift. To comment or ask technical questions about this book, send email to bookques‐ tions@oreilly.com. For more information about our books, courses, conferences, and news, see our web‐ site at http://www.oreilly.com. Find us on Facebook: http://facebook.com/oreilly Follow us on Twitter: http://twitter.com/oreillymedia Watch us on YouTube: http://www.youtube.com/oreillymedia Acknowledgments Mars would like to thank her family and coauthors for being supportive even when/ though she’s intolerable, and the University of Tasmania for all the opportunities it has afforded her. xvi | Preface

📄 Page 19

She wishes to particularly acknowledge educators Dr. Julian Dermoudy for making her fall in love with programming when she first started only three years ago, and Nicole Herbert for providing—in her role as a highly effective degree coordinator— the support and confidence she needed to do something with this new passion, along with the Australian Apple developer community and the AUC that gifted her their knowledge and interest in Swift, and her parents who continue to be her primary role models of character. Jon thanks his mother, father, and the rest of his crazily extended family for their tre‐ mendous support. Paris thanks his mother, without whom he wouldn’t be doing anything nearly as interesting, let alone writing books, and his partner (and lead author) Mars, as well as all his friends (several of whom he is lucky enough to have written this book with!). Tim thanks his parents and family for putting up with his rather lackluster approach to life. We’d all like to thank Michele Cronin and Rachel Roumeliotis, whose skills and advice were invaluable to completing the book. We’re really excited to work on more projects with both of you in the future! Likewise, all the O’Reilly Media staff we’ve interacted with over the course of writing the book have been the absolute gurus of their fields. A huge thank you to Tony Gray and the Apple University Consortium (AUC) for the monumental boost they gave us and others listed on this page. We wouldn’t be writ‐ ing this book if it weren’t for them. And now you’re writing books, too, Tony—sorry about that! Thanks also to Neal Goldstein, who deserves full credit and/or blame for getting us into the whole book-writing racket. We’re thankful for the support of the goons at MacLab (who know who they are and continue to stand watch for Admiral Dolphin’s inevitable apotheosis), as well as pro‐ fessor Christopher Lueg, Dr. Leonie Ellis, and the rest of the current and former staff at the University of Tasmania for putting up with us. Additional thanks to Dave J., Jason I., Nic W., Andrew B., Jess L., and everyone else who inspires us and helps us. And very special thanks to the team of hard-working engineers, writers, artists, and other workers at Apple, without whom this book (and many others like it) would not have reason to exist. Thanks also to our tech reviewers, with special thanks to Chris Devers, Dominic Monn, and Nik Saers for their thoroughness and professionalism. Finally, thank you very much for buying our book—we appreciate it! And if you have any feedback, please let us know. Preface | xvii

📄 Page 20

(This page has no text content)