Learn Python Generative AI Journey from autoencoders to transformers to large language models (Zonunfeli Ralte, Indrajit Kar) (Z-Library)

Learn Python Generative AI Journey from autoencoders to transformers to large language models Zonunfeli Ralte Indrajit Kar www.bpbonline.com

First Edition 2024 Copyright © BPB Publications, India ISBN: 978-93-55518-972 All Rights No part of this publication may be reproduced, distributed or transmitted in any form or by any means or stored in a database or retrieval system, without the prior written permission of the publisher with the exception to the program listings which may be entered, stored and executed in a computer system, but they can not be reproduced by the means of publication, photocopy, recording, or by any electronic and mechanical means. LIMITS OF LIABILITY AND DISCLAIMER OF WARRANTY The information contained in this book is true to correct and the best of author’s and publisher’s knowledge. The author has made every effort to ensure the accuracy of these publications, but publisher cannot be held responsible for any loss or damage arising from any information in this book. All trademarks referred to in the book are acknowledged as properties of their respective owners but BPB Publications cannot guarantee the accuracy of this information.

www.bpbonline.com

Dedicated to My beloved Parents: R. Zohmingthanga and Sangthangseii – Zonunfeli Ralte My beloved Parents: Avijit Kar and Puspa Kar – Indrajit Kar

About the Authors Zonunfeli Ralte, a seasoned professional with a Master’s in Business Administration and Economics, boasts 15 years of experience in Analytics, Finance, and AI. Currently, she is the CEO and Founder of RastrAI, while also serving as a Principal AI Consultant, developing GenAI applications for diverse industries. Zonunfeli has an impressive academic contribution with 6 IEEE research papers including Large Language Models Deep learning and computer vision, 3 of which received best paper awards. Her multifaceted expertise and leadership make her a notable figure in the AI community. Additionally, she has filed 1 patent in GenAI. Indrajit Kar, a master’s graduate in Computational Biology from Bengaluru, also holds a Bachelor’s in Science from the same institution with more than two decades of experience in AI and ML. He is an experienced intrapreneur, having built AI teams at Siemens, Accenture, IBM, and Infinite Data Systems. Presently, he is the AVP and Global Head of AI and ML leading AI research (ZAIR) and Data Practices. Indrajit has published 22 research papers across IEEE, Springers, Wiley Online Library, and CRC press, covering topics like LLM, Computer Vision, NLP, and more. He has 14 patents, including Generative AI. He is a mentor for startups and a recipient of multiple awards, including the 40 Under 40 Data Scientists award. He is also author of 2 AI books.

About the Reviewers ❖ Utkarsh Mittal is a Machine Learning manager at Gap Inc., a global retail company. He has more than ten years of practice experience in machine learning automation and is a leader of big AI-based database projects. He received his Master’s in Industrial Engineering with a Supply Chain and Operations Research major from Oklahoma State University, USA. He is closely associated with research groups and editorial boards of high-profile International Journals and research organizations. He is passionate about solving complex business challenges and encouraging innovation through upcoming technologies. He is a Senior member of the IEEE Computer Society. ❖ Arun wears many hats, but they all share a common thread - a love for building, learning, and driving meaningful impact. As a Senior Product Engineer at a Big 4 Firm, his eight years of experience translate into a potent blend of expertise. He’s the go-to for crafting and deploying Machine Learning pipelines on AWS, wielding frameworks like Kubernetes and Sagemaker with masterful hands. Arun’s passion extends beyond code. He’s an avid learner, constantly upskilling in MLOps and beyond. This thirst for knowledge led him to explore the fascinating world of Large Language Models. Under his guidance, teams have crafted projects that revolutionized document summarization for EdTech, unearthed customer sentiment like a treasure hunter, and even built an AI code assistant that boosts programmer productivity.

Acknowledgements Zonunfeli Ralte: To my family, especially my parents, sisters, and cousins, your unwavering encouragement and belief in my abilities have been the bedrock of my journey. Your support has been a guiding light, empowering me to pursue and accomplish this endeavor. I also wish to express my heartfelt appreciation to the people of Mizoram, particularly the community of Ramthar Veng. Our rich culture and vibrant spirit have been a constant source of motivation and have deeply influenced my perspectives and writing. A special acknowledgment goes to BPB Publications and my co-author Indrajit Kar for their patience and trust in my vision. Your flexibility in allowing the book to be published in multiple parts has been crucial in adequately covering the expansive and evolving field of AI. Lastly, I thank my companies for providing an environment that fosters learning and growth. The opportunities to explore and develop GenAI applications have been fundamental in accumulating the knowledge shared in this book. To all, your hidden and visible support has shaped this journey in countless ways, and for that, I am forever grateful.

Indrajit Kar: I extend my deepest appreciation to my family, particularly my parents, wife, in-laws and children, whose steadfast encouragement and unwavering belief in my abilities have formed the cornerstone of my journey. Your support has illuminated my path, empowering me to pursue and fulfill this endeavor with confidence and dedication. I must also express my profound gratitude to BPB Publications for their patience and trust in my vision. Their flexibility in allowing this book to be published in multiple segments has been pivotal in thoroughly addressing the broad and dynamic landscape of AI. Furthermore, I am immensely thankful to my companies for creating an environment that nurtures learning and growth. The opportunities they have provided to delve into and develop GenAI applications have been instrumental in gathering the insights shared in this book. To everyone involved, both in visible and unseen ways, your support has profoundly shaped this journey. For this, I am eternally grateful.

Preface Learn Python Generative AI is an extensive and comprehensive guide that delves deep into the world of generative artificial intelligence. This book provides a thorough understanding of the various components and applications in this rapidly evolving field. It begins with a detailed analysis, laying a solid foundation for exploring generative models. The combination process of different generative models is discussed in depth, offering a roadmap to understand the complexities involved in integrating various AI models and techniques. The early chapters emphasize the refinement of TransVAE, an advanced variational autoencoder, showcasing improvements in its encoder-decoder structure. This discussion sets the stage for a broader examination of the evolution of AI models, particularly focusing on the incorporation of the SWIN-Transformer in generative AI. As the book progresses, it shifts focus to the practical applications of generative AI in diverse sectors. In-depth chapters explore its transformative potential in healthcare, including applications in hospital settings, dental, and radiology, underscoring the impact of AI in medical diagnostics and patient care. The role of GenAI in retail and finance is also thoroughly examined, with a special emphasis on corporate finance and insurance, demonstrating how AI can revolutionize customer engagement, risk assessment, and decision-making.

Each sector-specific chapter is enriched with real-world examples, challenges, and innovative solutions, offering a comprehensive view of how generative AI is reshaping various industries. The concluding chapters synthesize the key learnings from all topics, providing insights into the future trajectory of generative AI. Chapter 1: Introducing Generative AI - The objective of this chapter is to provide a comprehensive understanding of generative models, including an overview of generative models, a comparison of discriminative vs generative models, an introduction to the types of discriminative and generative models, as well as their strengths and weaknesses. By the end of the content, readers should be able to differentiate between discriminative and generative models, understand the different types of each, and make informed decisions about which type of model is most appropriate for their needs. Chapter 2: Designing Generative Adversarial Networks - In this chapter, the objective is to delve into the multifaceted landscape of GANs by comprehensively exploring various types of GANs and their intricate architectures. By the end of this chapter, readers will be equipped with a solid understanding of the architecture, equations, and crucial design factors associated with different GAN variants. The chapter will dissect discriminator and generator losses, shed light on pivotal GAN types, including Vanilla GAN, Deep Convolutional GAN, Wasserstein GAN, Conditional GAN, CycleGAN, Progressive GAN, StyleGAN, and Pix2Pix, and address the major challenges encountered in designing effective GAN architectures. Through an in-depth analysis of each architecture, readers will gain the knowledge necessary to make informed decisions when selecting and designing GANs for various generative tasks.

Chapter 3: Training and Developing Generative Adversarial Networks - The objective of this book chapter is to provide readers with a comprehensive understanding of the process of training and tuning GANs, including the latest techniques and best practices for improving the stability and performance of GAN models. Chapter 4: Architecting Auto Encoder for Generative AI - The primary goal of this chapter is to explore the fascinating world of autoencoders in the context of generative AI. We will delve into the inner workings of autoencoders, discussing their architectural variations, training strategies and their applications in generating diverse and high-quality outputs across various domains. Furthermore, we will examine advanced techniques that leverage autoencoders, such as Variational AutoEncoders (VAE) and Generative Adversarial Networks (GAN), which push the boundaries of generative AI even further. Throughout this chapter, and the next, we will also discuss the key challenges associated with autoencoders for generative tasks, including issues like mode collapse, blurry outputs, and training instability. We will explore solutions and strategies to mitigate these challenges, providing practical insights and recommendations for building robust and effective generative models using autoencoders. By the end of this chapter, readers will have gained a comprehensive understanding of autoencoders as a powerful tool in the realm of generative AI. They will have a solid grasp of the fundamental concepts, practical considerations, and cutting-edge advancements that can enable them to apply autoencoders effectively in their own projects and unlock the potential of generative models to create realistic and novel outputs.

Chapter 5: Building and Training Generative Autoencoders - The key objectives of this chapter are to provide the reader with a deep understanding of autoencoders and their applications. By the end of this chapter, readers will gain a comprehensive understanding of the concept of latent space and its significance in autoencoders, explore the concept of dual input autoencoders and their usefulness in handling missing values and multi-modal data, and familiarize themselves with various loss functions commonly used in autoencoders and their role in training and reconstruction. The readers will also learn about potential issues during training, such as overfitting, vanishing gradients, and noisy data, along with strategies to mitigate them, discover optimization techniques specific to autoencoders for effective model training and performance enhancement, as well as understand the differences between autoencoders and variational autoencoders and their respective benefits. Lastly, the reader will acquire the knowledge and skills to leverage autoencoders in practical scenarios for data representation, generation, and anomaly detection. Chapter 6: Designing Generative Variation Auto Encoder - By the end of this chapter, the reader will be able to understand the fundamental differences between VAEs and traditional AEs. We will also explore the network architecture of VAEs, including the encoder and decoder networks, and their role in learning latent representations. The reader will also gain insight into the mathematical principles underlying VAEs, including the reparameterization trick and the ELBO objective function.

The chapter will then move to advanced techniques in VAEs, such as employing different prior distributions, utilizing various forms of the encoder network, and handling missing or incomplete data. We will also discover methods for interpreting the latent space of a VAE and visualizing its representations, explore the generative capabilities of VAEs by generating novel samples using the decoder network, and lastly, acquire the necessary knowledge and skills to apply VAEs in practical applications, including image generation, natural language processing, and anomaly detection. By achieving these key objectives, readers will develop a comprehensive understanding of VAEs and be able to leverage their power and flexibility in various domains, ultimately enhancing their ability to learn and generate meaningful representations from complex data. Chapter 7: Building Variational Autoencoders for Generative AI - By the end of this chapter, the reader will have explored various architectural choices, including convolutional or Non convolution networks, to handle complex dependencies in VAEs. We will also investigate the impact of KL divergence and different prior distributions on the generative process of VAEs, and develop strategies to effectively handle missing or incomplete data within the VAE framework. The reader will also understand the role of loss functions and address potential issues during training to ensure stable convergence, as well as optimize VAE performance and generative capabilities for diverse data modalities. By achieving these key objectives, readers will develop a comprehensive understanding of VAEs and be able to leverage their power and flexibility

in various domains, ultimately enhancing their ability to learn and generate meaningful representations from complex data. Chapter 8: Fundamental of Designing New Age Generative Vision Transformer - By the end of this chapter, readers will have a solid understanding of transformers, their underlying principles, and their various applications in natural language processing and computer vision. They will also have the necessary knowledge to build, train, and fine-tune transformer models for their own use cases. The readers will gain a comprehensive introduction to transformers as a class of neural networks. This includes explaining their significance in revolutionizing natural language processing and their current applications in computer vision. Then, we will explore fundamental transformer concepts, delve into the basic principles and key components of transformers, such as self- attention mechanisms and the transformer architecture. This chapter will cover generative transformers and highlight the main differences between regular transformers and those designed for generative tasks. Apart from this, the reader will also be able to analyze different types of attention, such as self-attention, cross-attention, and multi-headed attention, and elucidate their specific applications in image processing. Lastly, we will explore transformer math and positional encoding. Chapter 9: Implementing Generative Vision Transformer - In this chapter, our primary objective is to explore and understand the fundamental distinctions between Generative Transformers and conventional Transformers, highlighting their key differences and applications within the realm of image generation. We will then delve into VAE models and their application to the STL dataset, emphasizing their capability to capture latent features and generate images. Building upon this

foundation, our objective further extends to the conversion of a VAE model into a Generative Transformer model, showcasing the integration of these two powerful architectures to enhance image synthesis. Throughout the chapter, we will compare Generative Transformers and Transformers. We will thoroughly dissect the distinctions between Generative Transformers and traditional Transformers in terms of architecture, training methodologies, and their respective strengths and weaknesses. We’ll construct VAEs for the STL dataset, then transition to Generative Transformer models, adapting VAE components to fit Transformer’s self-attention and positional encodings. Our comprehensive evaluation will compare image quality, diversity, and speed against traditional models. We’ll also explore real-world applications, demonstrating the model’s capability to produce diverse, contextually coherent images. Ultimately, this chapter aims to deepen understanding of Generative Transformers versus traditional models, guide in VAE construction, and reveal the innovative transition to Generative Transformer architecture. Chapter 10: Architectural Refactoring for Generative Modeling - In this chapter, our primary objective is to explore the combination process, and delve into the process of synergistically combining an encoder-decoder architecture with a transformer model for enhanced generative modeling in computer vision. We will investigate how to enhance the transformer model by introducing modifications and optimizations, contributing to improved performance and suitability for specific tasks, and provide an in- depth exploration of the SWIN transformer implementation, including detailing its architecture, components, and distinctions from other transformer variants.

Moreover, this chapter will introduce readers to advanced concepts encompassing combining hyper parameter tuning and model refactoring and aims to equip readers with a comprehensive understanding of the entire process, encompassing motivations for combining architectures, technical implementation details, and an appreciation of the intricacies of the SWIN transformer model. Through this holistic approach, readers will gain both theoretical insights and practical skills, setting the stage for innovative generative modeling using combined encoder-decoder-transformer architectures. Chapter 11: Major Technical Roadblocks in Generative AI and Way Forward - The designated sections of this chapter aim to unravel the challenges and innovative solutions in the fields of data representation, retrieval, and cross-modal understanding. Obstacles and technical hurdles delve into the multifaceted challenges faced in various domains, such as generative AI and computer vision. Text and image embeddings provide insights into the pivotal role of embeddings in transforming textual and visual data into condensed, meaningful vectors. It examines how embeddings facilitate the understanding of semantic relationships and contextual nuances within language and images. The objective is to showcase how embeddings bridge the gap between raw data and AI models, contributing to better comprehension, representation, and manipulation of diverse data types. Vector databases delves into the construction and application of databases where items are represented as vectors. The section emphasizes efficient retrieval through indexing, particularly similarity searches. It aims to elucidate the construction of structures that enable quick and accurate

querying of semantically related items, illustrating their significance in real-world applications. Image-to-image search utilizing the liberated pinecone vector databases explores the practical implementation of vector databases for image search tasks. It sheds light on the liberation of these databases for open exploration and outlines how they power efficient image retrieval mechanisms. This section aims to demonstrate how vector databases can revolutionize image search, transforming the way users discover visually similar content across a spectrum of applications. Chapter 12: Overview and Application of Generative AI Models - In this chapter, we embark on a journey through the dynamic landscape of technology’s role in various industries, without delving into complex code or algorithms. Imagine a world where cutting-edge innovations like LLM and Gen AI are not just buzzwords but integral tools reshaping healthcare, retail, finance, and insurance. The story begins in healthcare, where LLM streamlines compliance, analyzes intricate medical documents, and guides professionals through complex regulatory mazes. Meanwhile, Gen AI steps in to provide personalized medical advice, automate appointment scheduling, and deliver vital information to patients and healthcare providers, ensuring the highest quality of care. Transitioning to the retail sector, LLM ensures contractual accuracy, compliance, and vendor agreement efficiency. Gen AI transforms the customer experience, captivating shoppers with personalized recommendations and dynamic marketing strategies, creating a retail environment tailored to each individual. In the financial realm, LLM takes center stage, enhancing risk assessment, detecting fraud, and analyzing contracts with unparalleled precision. Simultaneously, Gen AI

optimizes customer service through AI-powered chatbots and virtual assistants, providing real-time and context-aware responses to financial inquiries. Finally, in the insurance sector, LLM drives claims efficiency, fraud detection, and regulatory compliance. Gen AI revolutionizes insurance by reshaping underwriting processes, crafting personalized policy offerings, and elevating customer interactions. Chapter 13: Key Learnings - The objective of this chapter is to synthesize and distill the core teachings and insights from chapters one through twelve. It aims to provide readers with a comprehensive summary, highlighting the key concepts, important takeaways, and significant learnings obtained from each preceding chapter. By consolidating this knowledge, the chapter seeks to offer a holistic understanding of the subject matter, reinforcing key ideas, and preparing readers for further exploration or application of the discussed principles. Ultimately, the objective is to enhance comprehension, retention, and practical application of the cumulative wisdom acquired throughout the previous chapters.