The AI Pocket Book (Emmanuel Maggiori) (z-library.sk, 1lib.sk, z-lib.sk)

Emmanuel Maggiori The AI Pocket Book M A N N I N G

AI sometimes “hallucinates.” If we want to use it effectively, we must be aware of its limitations and be ready to tackle them.

The AI Pocket Book

MANN I NG Shelter ISland Emmanuel Maggiori The AI Pocket Book

For online information and ordering of this and other Manning books, please visit www.manning.com. The publisher offers discounts on this book when ordered in quantity. For more information, please contact Special Sales Department Manning Publications Co. 20 Baldwin Road PO Box 761 Shelter Island, NY 11964 Email: orders@manning.com © 2025 Manning Publications Co. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or trans- mitted, in any form or by means electronic, mechanical, photocopying, or otherwise, without prior written permission of the publisher. Many of the designations used by manufacturers and sellers to distinguish their prod- ucts are claimed as trademarks. Where those designations appear in the book, and Manning Publications was aware of a trademark claim, the designations have been printed in initial caps or all caps. Recognizing the importance of preserving what has been written, it is Manning’s policy to have the books we publish printed on acid- free paper, and we exert our best efforts to that end. Recognizing also our responsibility to conserve the resources of our planet, Manning books are printed on paper that is at least 15 percent recycled and processed without the use of elemental chlorine. ∞ Manning Publications Co. 20 Baldwin Road PO Box 761 Shelter Island, NY 11964 ISBN 9781633435759 Printed in the United States of America The author and publisher have made every effort to ensure that the information in this book was correct at press time. The author and publisher do not assume and hereby disclaim any liability to any party for any loss, damage, or disruption caused by errors or omissions, whether such errors or omissions result from negligence, accident, or any other cause, or from any usage of the information herein. Development editor: Ian Hough Technical editor: Artur Guja Review editor: Radmila Ercegovac Production editor: Andy Marinkovich Copy editor: Lana Todorovic-Arndt Proofreader: Keri Hales Typesetters: Bojan StojanoviÊ and Tamara ŠveliÊ SabljiÊ Cover designer: Marija Tudor

v Development editor: Ian Hough Technical editor: Artur Guja Review editor: Radmila Ercegovac Production editor: Andy Marinkovich Copy editor: Lana Todorovic-Arndt Proofreader: Keri Hales Typesetters: Bojan StojanoviÊ and Tamara ŠveliÊ SabljiÊ Cover designer: Marija Tudor brief contents 1 ■ How AI works 1 2 ■ Hallucinations 59 3 ■ Selecting and evaluating AI tools 82 4 ■ When to use (and not to use) AI 98 5 ■ How AI will affect jobs and how to stay ahead 117 6 ■ The fine print 146

vi contents preface x acknowledgments xiii about this book xv about the author xviii about the cover illustration xix 1 How AI works 1 How LLMs work 2 Text generation 4  ■  End of text 5  ■  Chat 5 The system prompt 7  ■  Calling external software functions 8  ■  Retrieval-augmented generation 12 The concept of tokens 13 One token at a time 14  ■  Billed by the token 16 What about languages other than English? 16 Why do LLMs need tokens anyway? 18 Embeddings: A way to represent meaning 19 Machine learning and embeddings 20 Visualizing embeddings 21  ■  Why embeddings are useful 22  ■  Why LLMs struggle to analyze individual letters 23

contents vii The transformer architecture 25 Step 1: Initial embeddings 28  ■  Step 2: Contextualization 29  ■  Step 3: Predictions 32 Temperature 33  ■  Can you get an LLM to always output the same thing? 34  ■  Where to learn more 35 Machine learning 36 Deep learning 37  ■  Types of machine learning 38 How LLMs are trained (and tamed) 45  ■  Loss 47 Stochastic gradient descent 48 Convolutions (images, video, and audio) 51 Multimodal AI 53 No free lunch 56 2 Hallucinations 59 What are hallucinations? 60 Made-up facts 60  ■  Misinterpretation 62 Incorrect solutions to problems 63 Overconfidence 65  ■  Unpredictability 66 Why does AI hallucinate? 67 Inadequate world models 68  ■  World models: Theory vs. practice 69  ■  Misaligned objectives 70 Toy hallucination example: Price optimization 72 Will hallucinations go away? 74 Mitigation 75 Hallucinations can kill a product 78 Living with hallucinations 80 3 Selecting and evaluating AI tools 82 Proprietary vs. open source 83 How to decide 84 Off-the-shelf vs. fine-tuning 85 How to decide 88

contentsviii Customer-facing AI apps vs. foundation models 89 How to decide 89 Model validation, selection, and testing 90 Training set 90  ■  Validation set 91  ■  Test set 92 Performance measures 93 Accuracy 93  ■  Precision and recall 94  ■  Mean absolute error and root mean squared error 96 4 When to use (and not to use) AI 98 Building an AI-based product 99 Am I putting AI before the customer? 99 Are hallucinations okay? 102  ■  Do I need to explain how the output is generated? 103 Using conversational AI as an assistant 105 Can I describe the task succinctly and validate the output easily? 105  ■  Has anyone done it before? 107  ■  What does an excellent job look like? 110 Building LLM wrappers 112 Will users interact with my product using natural language? 113 5 How AI will affect jobs and how to stay ahead 117 Excellence gap 118 Excellence gap in software engineering 122 Recommendations 124 Stringent validation 126 Validation in software engineering 131 Recommendations 133

contentscontents ix Tight control 136 Control in software 139  ■  Recommendations 140 A new opportunity: Making the web more human 141 Philosophical detour: Automation and mass unemployment 142 6 The fine print 146 Copyright 147 Economics of AI 151 Smoke and mirrors 154 Regulation 157 Prohibited AI practices 157  ■  High-risk systems 158  ■  Transparency obligations 158 Foundation models 158 Resource consumption 159 Brains and consciousness 162 appendix A Catalog of generative AI tools 167 index 175

x preface In the 2010s, a methodology known as machine learning became extremely popular. The novelty of machine learning was that, instead of writing every detail of a computer program by hand, some parts were determined automatically by having a computer analyze data. While machine learning wasn’t new, it rose to prominence during this period thanks to increased computing power and an unprecedented amount of data ready to be exploited. Machine learning soon became the favorite methodology of artificial intelligence, which is a more general research field that tries to have computers perform tasks similarly to humans. Notably, AI researchers used machine learning to reach record performance in automated analysis of images, video, and text. They also used machine learning to build the famous game-playing software AlphaGo, which beat a human player at the difficult game of Go. Machine learning also boomed in the business world. For example, companies started using it to automatically analyze online shoppers’ data and generate personalized product recommendations.

preface xi Due to machine learning’s success and wide adoption in the AI field, people soon started using the terms “machine learn- ing” and “AI” interchangeably. The business world became highly enthusiastic about AI’s prospects and made big prom- ises. However, while AI expanded steadily in academia and business, it was not massively adopted by the general public. This was probably because general-purpose AI tools weren’t all that useful yet (think of Alexa and Siri) and because AI was still not that great at analyzing natural language. But in the late 2010s and early 2020s, a series of methodolog- ical innovations made AI much better at analyzing written lan- guage and generating new content. This led to a race to build AI tools that could be used as high-performing assistants by the general public. AI exploded in 2022, with the launch of a number of remark- able customer-facing AI apps. One of them was ChatGPT, which reached a hundred million users in three months. Another one was Midjourney, a powerful tool for creating realistic images from a textual description. Enthusiasm about AI soared and so did dramatic predictions about its effects. Some people predicted extreme productiv- ity gains. Others predicted massive unemployment due to AI replacing people’s jobs. In particular, many people argued that software engineers would become obsolete. I’m a software engineer who specializes in AI. I did my PhD in AI and have been involved in the field for over a decade. Early in my career, while I was impressed by AI, I became a bit frustrated by the amount of hype around it—I kept stumbling upon failed AI projects that were swept under the rug, and I had the impression that AI’s limitations were often overlooked. In 2023, I published a book on the subject, titled Smart Until It’s Dumb (Applied Maths Ltd, 2023). As opposed to other books on AI, which were either very positive or negative about it, I wanted to share a more nuanced view. As the title implies, I think AI can be really cool sometimes, but it can be less cool

prefacexii other times—think of those pesky hallucinations that AI often suffers from. After I wrote that book, people started asking me ques- tions about all things AI related. For example, they asked me whether I thought machines would become conscious or whether self-driving cars would soon roam every street. But the most common topic was the future of work. Specifically, aspir- ing software engineers seemed particularly concerned about their future careers. People asked me, “Is it even worth becom- ing a software engineer, now that AI can code?” A teacher told me a few of her students had dropped out because they thought AI would make their skills irrelevant. In addition, numerous software engineers started to use AI at work and build AI-based products, but they often told me they couldn’t make it work as intended. For example, they said AI often generated inconsis- tent outputs, and users didn’t appreciate it. This book is intended to help you ride the AI revolution, both in terms of using AI effectively and making sure your job stays ahead of what AI can do. The book is based on my own experience in the AI field and also on the numerous conver- sations I’ve had with people about it. You’ll read stories, reflec- tions, and general advice, which I hope you’ll find useful. After you finish the book, I hope you’ll feel that you under- stand AI better, including its limitations, and that you’ll dis- cover new ways of using AI effectively and future-proof your career against it.

xiii preface acknowledgments The most difficult thing about writing a book is not putting words together or thinking about grammar (which AI is quite good at). Instead, the most difficult thing is writing a book whose content resonates with the target audience. That’s why my biggest thank you goes to the humans who went through this book’s draft and shared useful advice to improve it. This includes my developmental editor at Manning, Ian Hough and my technical editor, Artur Guja, risk manager, computer scientist, systems developer, and financial markets professional with over 20 years of experience in the banking sector. I’d also like to thank my acquisitions editor, Andy Wal- dron, and the wider Manning team who’ve been extremely helpful throughout the process. Finally, many thanks to all the reviewers from the software industry who read the draft early on and shared their thoughts: Aarohi Tripathi, Aayush Bhutani, Aeshna Kapoor, Ajay Tan- ikonda, An Nadein, Anil Kumar Moka, Annie Taylor Chen, Anupam Mehta, Arpankumar Patel, Arpit Chaudhary, Ashish Anil Pawar, Batul Bohara, Devendra Singh Parmar, Divakar Verma, Gajendra Babu Thokala, Harsh Daiya, Karthik Rajashe- karan, Lalit Chourey, Maksym Prokhorenko, Manohar Sai Jasti,

acknowledgmentsxiv Martin Knudsen, Meghana Puvvadi, Mohit Palriwal, Naresh Dulam, Natapong Sornrpom, Nilesh Charankar, Nupur Baghel, Prachit Kurani, Prakash Reddy Putta, Prasann Pradeep Patil, Premkumar Reddy, Raghav Hrishikeshan Mukundan, Radhika Kanubaddhi, Rajeev Reddy Vishaka, Rajesh Daruvuri, Ram Kumar Nimmakayala, Riddhi Shah, Ruchi Agarwal, Sai Chiligireddy, Shivendra Srivastava, Siddharth Parakh, Subba Rao Katragadda, Sudheer Kumar Lagisetty, Sumit Dahiya, Sud- harshan Tumkunta, and Vishnu Challagulla. Your feedback helped improve this book. Thank you all!

xv acknowledgments about this book This book will help you navigate the AI revolution, using AI effectively in your work and making sure your job won’t be replaced by AI. The book was primarily written for software engineers, but its content was designed to be accessible to other audiences, too. So, there are no prerequisites to read this book, and anyone should be able to understand it. It is helpful, however, to know the basics of coding and math to fully understand all the examples. The book starts with a plain-English overview of how AI works. It then covers a wide range of timely and controversial AI-related topics such as hallucinations, the future of work, and copyright. Who should read this book? Two main groups of people should read this book. The first one is software engineers—aspiring, novice, and seasoned ones—who want to understand the effects of AI on their careers and prepare for it. The second group includes people related to or interested in the software industry, even if they’re not engineers them- selves. For example, these are product managers and startup

about this bookxvi entrepreneurs. One of this book’s reviewers said he thought the book would be useful not just for software engineers but also for “software sympathizers,” which I thought was a good way to put it. How this book is organized: A road map The book is divided into six chapters: ¡ Chapter 1: How AI works—This chapter explains how large language models and other types of AI work and how AI is built. ¡ Chapter 2: Hallucinations—This chapter explains the rea- sons for AI’s pesky mistakes (known as hallucinations), whether they will be fixed soon, and what we can do about them. ¡ Chapter 3: Selecting and evaluating AI tools—This chapter explains a method to select and compare different AI tools and avoid common biases in your evaluation. ¡ Chapter 4: When to use (and not to use) AI—This chapter is a checklist that will help you decide whether it is a good idea to use AI to assist you with a certain task or as the building block of a customer-facing product. ¡ Chapter 5: How AI will affect jobs and how to stay ahead—This chapter explains three characteristics of jobs that will help them resist AI advancements and how software engineers can stay relevant in the AI era. ¡ Chapter 6: The fine print—This chapter covers the less flat- tering side of AI, such as exaggeration, copyright disputes, and dubious comparisons of AI models with the human brain. It is meant to help you get up to speed with some of the bigger questions around AI. liveBook discussion forum Purchase of The AI Pocket Book includes free access to live- Book, Manning’s online reading platform. Using liveBook’s

about this bookabout this book xvii exclusive discussion features, you can attach comments to the book globally or to specific sections or paragraphs. It’s a snap to make notes for yourself, ask and answer technical questions, and receive help from the author and other users. To access the forum, go to https://livebook.manning.com/ book/the-ai-pocketbook/discussion. You can also learn more about Manning’s forums and the rules of conduct at https:// livebook.manning.com/discussion. Manning’s commitment to our readers is to provide a venue where a meaningful dialogue between individual readers and between readers and the author can take place. It is not a com- mitment to any specific amount of participation on the part of the author, whose contribution to the forum remains voluntary (and unpaid). We suggest you try asking the author some chal- lenging questions lest their interest stray! The forum and the archives of previous discussions will be accessible from the pub- lisher’s website for as long as the book is in print.

xviii about the author EmmanuEl maggiori, PhD, has been an AI industry insider for 10 years. He has developed AI for various applica- tions, from processing satellite images to packaging deals for holiday trav- elers. He is the author of the books Smart Until It’s Dumb, which analyzes the AI industry, and Siliconned, which analyzes the wider tech industry.