Share E-Book

Building Complex Multi-Agent Systems Using Pattern Prompting A guide to building robust and secure GenAI applications using… (Tim OBrien) (z-library.sk, 1lib.sk, z-lib.sk)

Author

Rating No ratings yet

Log in to rate

AI
Language English

Learn how to build GenAI applications using proven patterns and abstractions to work confidently across tools and frameworks Key Features • Learn core abstractions and enterprise patterns behind modern GenAI systems • Build secure, scalable agentic architectures using familiar engineering tools • Generate multi-agent workflows and RabbitMQ-based GenAI configurations • Use our Topologos prompt to easily build and test any agentic system Learn how to build GenAI applications using proven software engineering patterns instead of rapidly changing frameworks. This book helps engineers build secure, scalable agentic systems with familiar tools and practical, engineer-to-engineer architectural guidance. You will connect GenAI concepts such as agentic workflows, embeddings, and vector databases to enterprise patterns, including components, adapters, and microarchitectures. Established GoF and enterprise design patterns help explain agentic behavior and system design, enabling you to reason about architecture rather than memorize tools. The book also shows you how to generate multi-agent and GenAI patterns as RabbitMQ configurations for scalable orchestration and communication. Using language-agnostic examples and widely used messaging, orchestration, and data technologies, you will build production-ready systems that integrate with existing infrastructure without unnecessary complexity. You will also use our Topologos prompt to build, modify, and test deploy-ready multi-agent systems quickly while improving robustness and maintainability. By the end of this book, you will be able to design reliable GenAI systems, make informed architectural decisions, and adapt confidently as tools and frameworks evolve. Who this book is for This book is written for experienced software engineers, architects, and technical leads who want to add GenAI to their toolbox without abandoning the engineering principles that define good software.

Format PDF
Size 4.1 MB
5
Views
(First 20 pages)

Registered users can read the full content for free

Register as a Gaohf Library member to read the complete e-book online for free and enjoy a better reading experience.

Page 1
(This page has no text content)
Page 2
Building Complex Multi-Agent Systems Using Pattern Prompting A guide to building robust and secure GenAI applications using software engineering best practices Tim O'Brien
Page 3
Building Complex Multi-Agent Systems Using Pattern Prompting Copyright © 2026 Packt Publishing All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information. Portfolio Director: Gebin George Relationship Lead: Sonia Chauhan Project Manager: Prajakta Naik Content Engineer: Mark D'Souza Technical Editor: Rahul Limbachiya Indexer: Rekha Nair Production Designer: Prashant Ghare Growth Lead: Nimisha Dua First published: May 2026 Production reference: 1080526 Published by Packt Publishing Ltd. Grosvenor House 11 St Paul's Square Birmingham B3 1RB, UK. ISBN 978-1-80611-429-0 packtpub.com
Page 4
To John Milo Slick, former manager at Nortel Networks, who taught me how to write quality code and shared many valuable lessons about life. – Tim O'Brien
Page 5
Contributors About the author Tim O'Brien is a former Google engineer with over 22 years of experience designing enterprise systems for organizations including CIT Group, Bank of America, Tokyo Bank, and the Danish government. After leaving Google in 2017, he founded three successful AI startups and is currently building his latest venture, Quizitive. Based in Vietnam with his wife, Tim enjoys badminton, swimming, and learning new languages.
Page 6
About the reviewers Michael (Mike) Erlihson, PhD, is a prominent AI researcher and educator known for his rigorous mathematical approach to deep learning. He is the founder of the #DeepNightLearners community and the author of the Mathy AI platform, where he has authored more than 600 comprehensive reviews of deep learning research papers. His work distillates complex advancements in LLMs, state-space models, and neural architectures for thousands of practitioners and researchers worldwide. A seasoned communicator and thought leader, Michael is the co-host of two popular podcasts, DataScienceDecoded and Explainable, and is a frequent speaker at international tech conferences and meetups. He is also the author of the forthcoming book Deep Learning in Hebrew. Currently, Michael serves as the Head of AI at DriveNets, leading AI innovation for high-scale networking and cloud infrastructure. He holds a PhD in Applied Mathematics from the Technion – Israel Institute of Technology, with a specialization in probability theory and statistical mechanics. With over 20 years of experience in algorithm development, he brings a unique blend of theoretical depth and industrial leadership to the technical review process. Shivendra Srivastava is an engineering manager at AWS, where he leads teams building critical serverless infrastructure across AWS Lambda, Athena, Glue, and Amazon Bedrock. With a deep focus on sub-millisecond latency optimization and AI-driven cloud systems, he bridges the gap between cutting-edge research and production-scale engineering. He has published AI research with the IEEE Silicon Valley Chapter. A sought-after voice in the AI community, Shivendra judges AI hackathons and speaks at technology conferences. He is based in the Seattle, Washington area. Tam Nguyen is a senior AI engineer with a deep background in big data engineering and machine learning. Currently, his focus is on the frontier of generative AI, where he builds sophisticated agentic AI systems utilizing multi-agent architectures, long-term memory, and agent harness engineering. With extensive experience in LLMOps, MCP, and IDP, he bridges the gap between complex data architectures and production-ready AI agents. From designing data lakehouses to fine-tuning LLMs, he is dedicated to building the architecture and intelligence that power the next generation of A2A ecosystems.
Page 7
Nguyễn Minh Tâm is a senior full stack software engineer and AI researcher with a master's degree in computer science from Can Tho University. He combines deep theoretical knowledge in AI with extensive practical experience leading cross-functional teams as a technical leader. Tâm has a proven track record in building complex systems, from blockchain-based smart contracts to scalable web and mobile architectures. A former Top 10 Informatics Olympic medalist, he excels at solving sophisticated technical challenges and is dedicated to leveraging advanced computer science and AI to develop high-impact, innovative software solutions.
Page 8
Table of Contents Preface xix Free benefits with your book ............................................................................ xxvi Chapter 1: Introduction: Patterns, Abstractions, and the GenAI Landscape 1 A vocabulary bridge: GenAI terms and their IT equivalents .................................... 2 Studying GenAI .................................................................................................... 3 On the use of abstractions and patterns ................................................................. 5 How patterns are used in this book ....................................................................... 6 Benefits of breaking down GenAI microarchitectures into their constituent patterns .............................................................................................................................. 7 What is an LLM? ................................................................................................... 8 LLM as a continuation of big data • 9 LLM as a badly behaving RESTful endpoint • 9 How do LLMs work? ........................................................................................... 10 What is agentic AI? .............................................................................................. 12 Summary ............................................................................................................ 13 Chapter 2: Embeddings: The Language of AI 15 What are embeddings? ........................................................................................ 16 What do vectors look like? • 17 What is affinity? • 18 Calculating semantic similarity programmatically ............................................... 21 Selecting an embedding model ........................................................................... 22 Testing embeddings in your code ........................................................................ 23 What are vector databases? ................................................................................. 24 Chunking documents ......................................................................................... 26 Chunking decisions • 27
Page 9
Chunking strategies • 28 Summary ........................................................................................................... 30 Chapter 3: Building with GenAI: Parameters, Tuning, and Project Phases 33 Tuning GenAI systems: principles and practices .................................................. 34 Tuning the parameters of your GenAI project ...................................................... 35 1) Document → Chunking • 36 2) Chunking → Retrieval • 37 3) Retrieval ↔ Prompting • 37 4) Model → Context/Tokens → Prompts • 37 5) Temperature ties to prompt behavior • 37 Configuring a GenAI application to answer any question about the Harry Potter books ................................................................................................................... 38 Chunking strategies • 41 Temperature • 43 Best practices for building a GenAI project .......................................................... 46 Project initiation • 47 Intermediate goals • 47 Crossing the finish line • 48 Summary ........................................................................................................... 49 Chapter 4: Building Your First RAG App 51 What does "production-grade" software mean? .................................................. 52 Making technology choices: advantages of queues and topics .............................. 55 Why RabbitMQ for enterprise-grade GenAI systems • 56 Quick tour of RabbitMQ • 58 Building a production-class RAG application using RabbitMQ with integration patterns ............................................................................................................... 63 Building the LLM client ...................................................................................... 67 Template Method • 68 Adapter • 69 Strategy • 69 Table of Contents viii
Page 10
Generating code and deployment configuration .................................................. 70 Deploying our solution • 71 Summary ........................................................................................................... 72 Chapter 5: Starting Your Data Migration Project 75 Identifying documents to ingest into the vector database .................................... 76 Selecting the right ETL tool ................................................................................ 77 Planning your data pipeline ................................................................................ 80 Capacity planning and throughput • 80 Handling change and continuous operations • 81 Ensuring security, privacy, and compliance • 82 Managing parallel pipelines and CI/CD • 82 Testing and validation • 84 Cost optimization and throttling ........................................................................ 84 Evaluating retrieval quality ................................................................................ 85 Knowledge structure and advanced retrieval (optional) • 85 Hybrid search and the limits of pure vector retrieval • 86 The importance of data cleaning in GenAI pipelines • 87 Graph databases and the emergence of knowledge graphs • 88 Toward integrated knowledge retrieval • 89 Summary ........................................................................................................... 90 Chapter 6: Ingesting Data Using Airbyte and Pinecone 91 Analysis of applicable patterns ............................................................................ 92 Integration patterns implemented by ETL • 93 Channel adapter • 93 Strategy pattern • 94 Building our ETL pipeline ................................................................................... 95 Installation of Pinecone and Airbyte • 96 Analysis of documents • 96 Options for configuring Airbyte • 97 Configuring Airbyte using its no-code interface • 98 ix Table of Contents
Page 11
Importing a YAML manifest • 101 Summary ......................................................................................................... 102 Chapter 7: Tips and Best Practices 105 Tips and best practices to help make your GenAI project a success ..................... 106 Best practices for designing and operating GenAI systems • 106 Have an R&D mindset • 106 Don't be afraid to reason things out yourself • 106 Discover your own agentic patterns • 107 Lead with small POCs first – keep all POCs under 2 days • 108 Set latency and throughput targets and load test frequently • 108 Have a plan for managing drift • 108 Don't use data produced by GenAI with your other IT systems • 109 Validate your UX with extensive usability testing • 109 Efficient project management • 109 Security and data privacy • 111 Prompt injection attacks • 111 Data leakage through context windows • 112 Handling PII in prompts • 112 Output validation and harmful content filtering • 113 Compliance considerations by industry • 113 Building a security-first culture • 114 Cost management and token budgeting • 114 Building a cost model early • 115 Choosing the right model for each task • 116 Fine-tuning as a cost optimization strategy • 117 Setting cost alerts, budgets, and governance • 117 Vendor lock-in and model portability • 118 Prompt versioning and governance • 118 Treat prompts as code • 119 Adopt a versioning strategy • 119 Establish review and approval workflows • 119 Table of Contents x
Page 12
Enable prompt rollback • 119 Define ownership and accountability • 119 Evaluation and testing frameworks • 120 Building your evaluation dataset • 120 Automating evaluation pipelines • 120 Red-teaming your application • 120 Handling non-determinism in tests • 121 Testing for drift • 121 Summary .......................................................................................................... 121 Chapter 8: Pattern-Guided Coding: Using Patterns as the Design Vocabulary for GenAI Systems Built on RabbitMQ 123 The need for pattern-guided coding ................................................................... 125 Reason 1: Communication breaks down without a shared vocabulary • 126 Reason 2: Design decisions become invisible • 126 Reason 3: GenAI systems fail at the integration layer • 126 How pattern-guided coding works ..................................................................... 127 Building your app in a four-step Topologos process ............................................ 129 Phase 1: Clarification before design • 129 Phase 2: Pattern analysis across four lenses • 130 Phase 3: Iterative approval — one decision at a time • 131 Phase 4: Final topology output • 132 Building producers and consumers with GoF patterns ........................................ 135 The Strategy pattern for producer routing • 135 The Command pattern for agent task messages • 136 The Template Method pattern for consumer pipelines • 137 The Channel Adapter pattern for external integration • 137 Consumer acknowledgment and the manual ACK rule • 138 The multi-tier dead-letter queue strategy .......................................................... 138 Generating any known GenAI pattern with a single command ........................... 140 Getting ready to deploy • 140 The iterative approval loop • 142 xi Table of Contents
Page 13
Auditing an existing system • 142 Customizing your GenAI application .................................................................. 143 Case study: extending a single-LLM RAG to a dual-LLM Scatter-Gather ............... 143 The before topology: single LLM • 143 The Topologos session: proposing the change • 144 The new queue topology: dual LLM with Scatter-Gather • 146 What changed and what stayed the same • 146 The Aggregator's correlation window • 148 Summary .......................................................................................................... 151 Chapter 9: Implementing the ReAct Pattern Over RabbitMQ 153 Understanding the ReAct pattern ....................................................................... 154 Setting up the RabbitMQ topology ..................................................................... 155 Building the ReAct agent ................................................................................... 165 Message schemas • 165 The LLM call (_think) • 165 Tool dispatch (_dispatch_tool) • 166 The loop and ACK strategy • 166 Implementing tool workers ............................................................................... 177 Error handling in tool workers • 178 Scaling • 178 Sending commands to the agent ........................................................................ 185 Running the system end to end .......................................................................... 187 Setting up • 187 Start the workers • 188 Send a question • 188 Observing the dead letter queue ....................................................................... 189 Preparing the system for production .................................................................. 191 Summary .......................................................................................................... 192 Chapter 10: The Future and Limitations of LLMs 195 LLMs actually do: a precise description .............................................................. 196 Table of Contents xii
Page 14
The stochastic parrot: fluency without understanding • 197 Chomsky and the limits of statistical learning • 198 Searle's Chinese room: syntax is not semantics • 198 Penrose, Gödel, and the mathematical limits of computation • 199 The reasoning gap: what LLMs cannot reliably do ............................................. 200 Systematic mathematical reasoning • 200 Maintain logical consistency over long contexts • 200 Provide calibrated confidence in their responses • 200 Produce novel insights • 201 The symbol grounding problem • 201 What the hype gets wrong: a taxonomy of overclaiming .................................... 201 Why hype is harmful: the engineering costs ...................................................... 203 What LLMs are genuinely good at ..................................................................... 203 Thinking about the future with clear eyes ......................................................... 204 Practical guidance: working productively with known limitations .................... 205 Summary ......................................................................................................... 206 Appendix A: Pattern Reference: GoF, Enterprise Integration, Reliability, and GenAI Microarchitecture Patterns 207 Pattern index ................................................................................................... 208 Part 1 — GoF Design Patterns ............................................................................ 209 A.1 Strategy • 209 GoF — Behavioral | Chapters 4, 6 • 209 A.2 Adapter • 210 GoF — Structural | Chapters 4, 6 • 210 A.3 Template Method • 210 GoF — Behavioral | Chapter 4 • 210 Part 2 — Enterprise Integration Patterns ........................................................... 211 A.4 Message Channel • 211 EIP — Messaging Infrastructure | Chapters 4, 6 • 211 A.5 Channel Adapter • 212 EIP — Messaging Infrastructure | Chapters 4, 6 • 212 xiii Table of Contents
Page 15
A.6 Publish–Subscribe Channel • 212 EIP — Messaging | Chapter 4 • 212 A.7 Content Enricher • 213 EIP — Message Transformation | Chapters 3, 4 • 213 A.8 Request–Reply • 214 EIP — Messaging | Chapter 4 • 214 A.9 Correlation Identifier • 215 EIP — Messaging | Chapter 4 • 215 A.10 Scatter–Gather • 215 EIP — Message Routing | Chapter 4 • 215 A.11 Dead Letter Channel • 216 EIP — Messaging | Chapter 4 • 216 A.12 Message History • 217 EIP — System Management | Chapter 4 • 217 A.13 Content-Based Router • 217 EIP — Message Routing | Chapter 4 • 217 A.14 Pipes and Filters • 218 EIP — Message Routing | Chapters 5, 6 • 218 Part 3 — Reliability Patterns .............................................................................. 218 A.15 Circuit Breaker • 218 Reliability Pattern | Chapter 4 • 218 A.16 Retry with Exponential Backoff • 219 Reliability Pattern | Chapters 4, 7 • 219 Part 4 — Microarchitecture Patterns ................................................................. 220 A.17 Orchestration • 220 Microarchitecture Pattern | Chapters 1, 4, 7 • 220 A.18 Choreography • 220 Microarchitecture Pattern | Chapters 1, 7 • 220 A.19 RAG Microarchitecture • 221 GenAI Microarchitecture | Chapters 2, 3, 4, 5 • 221 Table of Contents xiv
Page 16
Appendix B: Topologos User Manual 225 Quick start ....................................................................................................... 226 Loading the prompt • 226 Starting a session • 226 Conversational • 226 Pattern-first • 226 Topology first • 227 Approve decisions one at a time • 227 Get your deployable artifact • 227 When to use Topologos ...................................................................................... 227 Core concepts ................................................................................................... 228 The two layers – pattern versus topology • 228 Cross-layer example • 229 The four-phase protocol • 229 Example Phase 1 acknowledgment (fast path) • 230 Example Phase 3 decision • 230 Command reference .......................................................................................... 231 All commands at a glance • 231 Session commands • 233 Example /status output • 233 Pattern layer commands • 234 Topology layer commands • 234 Definition commands • 234 Operational commands • 235 Example /validate output • 235 Example /simulate tool-timeout output • 235 Example /cost output • 236 Diagram commands • 236 Modifiers .......................................................................................................... 237 Auto-derived templates .................................................................................... 238 Generating the deployable artifact .................................................................... 239 xv Table of Contents
Page 17
End-to-end workflow • 239 The seven Phase 4 artifacts • 239 Running /deploy • 240 Targets • 240 Modifiers • 241 Examples • 241 Rendering diagrams to PNG • 242 Mermaid (default) • 242 Graphviz DOT • 242 SVG • 242 PlantUML • 242 Applying the manifest to a real broker • 242 Route 1 – rabbitmqadmin (CLI) • 242 Route 2 – Management HTTP API • 243 Route 3 – Terraform (recommended for production) • 243 Generating it inline – a typical full session • 243 Worked examples ............................................................................................. 244 Example A – Generating ReAct conversationally • 244 Phase 2 (excerpt) • 245 Phase 3 – eight decisions (selected highlights) • 245 Phase 4 manifest excerpt • 246 Example B – Composed pattern (ReAct + Tool Use) • 247 Invocation • 247 What changes vs. canonical ReAct? • 247 Why compose rather than extend • 247 Example C – Extending a canonical pattern (critic gate) • 247 Invocation • 248 Phase 1 acknowledgment • 248 Additional Phase 3 decisions • 248 Chaining multiple extensions • 248 Example D – Defining a new pattern (Debate-and-Adjudicate) • 249 Step 1 – invoke /define • 249 Table of Contents xvi
Page 18
Step 2 – Topologos validates and echoes a synthetic reference card • 249 Step 3 – compose, extend, persist • 250 When to define vs. extend vs. compose • 250 Example E – Defining an organization-wide topology archetype • 250 Scenario • 251 Step 1 – define the queue archetype • 251 Step 2 – define the DLQ topology • 251 Step 3 – define the security baseline • 252 Step 4 – bundle them as a topology archetype • 252 Step 5 – use it and lay a pattern on top • 252 Step 6 – persist across sessions • 252 The payoff • 253 Recipes ............................................................................................................. 253 Tips and best practices ...................................................................................... 255 Troubleshooting ............................................................................................... 255 FAQs ................................................................................................................. 257 Glossary ........................................................................................................... 259 Canonical pattern reference ............................................................................... 261 ReAct • 261 Plan-and-Execute • 261 Reflection • 262 Tool Use • 262 Multi-Agent Collaboration • 262 RAG • 262 Memory • 263 Orchestrator-Subagent • 263 Human-in-the-Loop • 263 Tree of Thoughts • 264 Topology-layer canonical primitives ................................................................. 264 Appendix C: Unlock Your Exclusive Benefits 267 Unlock this Book's Free Benefits in 3 Easy Steps ................................................. 268 xvii Table of Contents
Page 19
Other Books You May Enjoy 272 Index 275 Table of Contents xviii
Page 20
Preface We are living through one of the most significant technological shifts in the history of software engineering. GenAI has arrived with the force of a tidal wave, and like every major wave before it—the internet, cloud computing, and mobile—it brings both genuine opportunity and considerable hype. By 2025, the headlines were difficult to ignore: "95% of generative AI pilots at companies have already failed." Much of this failure stems from a large knowledge gap between engineers who know how to build production-quality software and specialists in AI and agentic AI. This book closes that gap from the engineering perspective by providing a concise survey of what engineers and architects need to begin building GenAI applications. By focusing on fundamentals, relating concepts to established engineering practices, and productively avoiding unnecessary terminology, it helps readers quickly become familiar with GenAI technologies and best practices. With engineers in mind, the book introduces a new methodology for working with coding agents called Pattern-Guided Coding (PGC). PGC reduces errors by communicating through well-known software patterns, especially those from the Gang of Four and Enterprise Integration Patterns books. Patterns provide precise engineering language designed to reduce miscommunication within software teams. Because coding agents are trained on these smaller, highly precise, and internally consistent datasets, rather than billions of arbitrary lines of code, hallucinations become far rarer and easier to detect. PGC includes a reusable prompt, or "skill," called Topologos, capable of generating secure, scalable, and reliable frameworks from a single command: /pattern react + tool-use with critic-gate, circuit-breaker high-throughput regulated multi-tenant retry-5 Key features of this book include building fully deployable, secure, and scalable agentic systems with RabbitMQ; providing engineer-to-engineer guidance focused on practical agentic architectures; and using Topologos prompts to build, modify, and test deployable multi-agent systems. Support for most agentic patterns is included, removing the need to adopt many of the frameworks currently on the market. The resulting code remains familiar to enterprise engineers because it follows established architectural standards.
The above is a preview of the first 20 pages. Register to read the complete e-book.

Recommended for You

Loading recommended books...
Failed to load, please try again later

Tip the Site

Scan the WeChat Pay or Alipay code to tip. No login required.

WeChat Pay
Alipay
← Back to List