Building Production AI Agents for the Web Design and orchestrate autonomous agents with MCP, Multi-Agent Patterns, and Harness… (Christoffer Noring) (z-library.sk, 1lib.sk, z-lib.sk)
AI
Design, orchestrate, and deploy scalable AI agents with real-world patterns, multi-agent systems, and evaluation frameworks. Key Features Filled with real-world agentic web application examples using LLMs, RAG, and tool calling Design and implement AI assistants with reasoning patterns like ReAct Architect scalable agent systems with planning, orchestration, and autonomy Book Description AI is no longer just about generating text. It’s about building autonomous, goal-driven systems that can reason, act, and collaborate. Hands-On AI Agents for the Web is your practical guide to designing and deploying agentic web applications powered by Large Language Models (LLMs). Starting with the fundamentals of agentic thinking and responsible AI, this book walks you through integrating LLMs into real-world web apps. You’ll progressively enhance your systems with tool calling, Retrieval-Augmented Generation (RAG), and advanced reasoning patterns like ReAct, before moving into the design of fully autonomous agents. As you advance, you’ll explore agent architectures, planning systems, orchestration, and multi-agent collaboration, learning how modern frameworks and protocols like Model Context Protocol (MCP) enable scalable, interoperable AI systems. The book emphasizes hands-on learning, with practical examples that demonstrate how to build AI assistants, coordinate multiple agents, and apply proven design patterns, while avoiding common pitfalls. By the end, you’ll have the skills to build production-ready, intelligent web experiences powered by autonomous agents. What you will learn Understand agentic thinking and responsible AI principles Integrate LLMs into web applications and build prompt-driven systems Enhance LLM capabilities with tool calling and external integrations Implement RAG pipelines for context-aware, accurate AI responses Implement planning systems for goal-driven agents Orchestrate autonomous agents and manage workflows at scale Identify agent design patterns…
6
Views
Registered users can read the full content for free
Register as a Gaohf Library member to read the complete e-book online for free and enjoy a better reading experience.
Page
1
B u ild in g P ro d u c tio n A I A g e n ts fo r th e W e b First Edition C hristoffer N oring EXPERT INSIGHT Building Production AI Agents for the Web Design and orchestrate autonomous agents with MCP, Multi-Agent Patt erns, and Harness Engineering Christoff er Noring Integrate LLMs into web apps and extend their capabilities with tool calling and external systems Build RAG pipelines that ground AI applications in relevant business and application data Use MCP to connect AI applications with tools, resources, and capabilities Build intelligent AI agents using ReAct, advanced agent architectures, and multi-agent patterns Test and evaluate LLMs and agents to improve reliability and accuracy Build an agent harness with the context, tools, guardrails, and infrastructure needed to take AI applications into production WHAT YOU WILL LEARN Adding an LLM to an application is easy. Building an intelligent application that can reliably use your data, call tools, make decisions, complete tasks, and operate in production is much harder. Building Production AI Agents for the Web is a practical guide to engineering modern AI-powered applications beyond the basic chatbot. You’ll start by integrating LLMs into web applications and progressively extend their capabilities with tool calling, RAG, and MCP. From there, you’ll learn how these building blocks come together to create AI assistants and agents capable of reasoning and taking action. You’ll explore the ReAct architecture, build your fi rst agentic application, and progress toward advanced agent architectures and multi-agent systems. You’ll learn proven agent patt erns and anti-patt erns that help you make bett er architectural decisions as your applications scale. In the later chapters, you’ll learn how to test LLMs and agents, manage APIs, deploy agentic applications, apply responsible AI practices, and build an agent harness that provides the structure and guardrails agents need to operate reliably. By the end, you’ll understand the complete journey from integrating your fi rst LLM to engineering production- ready intelligent applications that can retrieve information, use tools, reason, collaborate, and take action. www.packtpub.com Building Production AI Agents for the Web First Edition Get a free PDF of this book packtpub.com/unlock/9781806103331
Page
2
Building Production AI Agents for the Web First Edition Design and orchestrate autonomous agents with MCP, Multi-Agent Patterns, and Harness Engineering Christoffer Noring
Page
3
Building Production AI Agents for the Web First Edition Copyright © 2026 Packt Publishing All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information. Portfolio Director: Ashwin Nair Relationship Lead: Aaron Lazar Project Manager: Ruvika Rao Content Engineer: Runcil Rebello Technical Editor: Arjun Varma Copy Editor: Runcil Rebello Indexer: Tejal Soni Proofreader: Runcil Rebello Production Designer: Salma Patel Growth Lead: Anamika Singh First published: Sep 2026 Production reference: 1230926 Published by Packt Publishing Ltd. Grosvenor House 11 St Paul's Square Birmingham B3 1RB, UK. ISBN 978-1-80610-333-1 www.packtpub.com
Page
4
I'd like to dedicate this book to all my colleagues at Microsoft and GitHub. You've made the last 8 years of my career truly meaningful as we've created a lot of great things together and helped millions of developers. – Chris Noring
Page
5
Contributors About the author Christoffer Noring is a passionate developer and educator who specializes in modern web technologies and AI integrations and works as an engineer at Microsoft. He's also a tutor at the University of Oxford and is a published author on Angular, RxJS, generative AI, and now MCP. Christoffer has almost two decades of experience in software development and is a frequent speaker at tech conferences worldwide. According to his manager, his best quality is being able to break down complex technical concepts into simple, understandable terms. He hopes you agree! When not coding or writing, Christoffer is probably growing another user community, mentoring developers, or spending time with his family.
Page
6
About the reviewers Mark McDonagh is a technology executive and information security specialist with 25+ years' experience in software, cloud transformation, and cybersecurity governance. CEO of Marcus- Antonio.com, he also leads Governance and Information Security at Verodat, supporting secure, resilient AI-driven data platforms. Mark holds a degree in Computer Systems and postgraduate and Master's qualifications spanning Executive Management, Cybersecurity and Forensics, Advanced Digital Technologies, and AI/ML. He actively contributes to standards initiatives, most recently Cloud Security Alliance's AI initiatives, including AICM, TAISE and STAR AI. In his spare time, Mark also volunteers to help people in need through his work for the Legion of Mary. Tejul Pandit is a Senior Staff Machine Learning Engineer with over seven years of experience in the field. Holding a Master's degree in Artificial Intelligence from Northwestern University, she is the author of multiple research papers and patents. Tejul excels in designing high- impact ML architectures, having previously led a landmark, large-scale machine learning deployment that drastically reduced human effort. She is currently driving enterprise AI innovation by building a sophisticated agent orchestrator to automate and resolve PANW firewall issues. Furthermore, Tejul is actively engaged in LLM-based Retrieval-Augmented Generation (RAG) research, developing efficient pipelines and prompting strategies, and frequently shares her expertise on enterprise AI applications at industry conferences.
Page
7
(This page has no text content)
Page
8
Table of Contents Preface xxiii Free benefits with your book ................................................................................................. xxx Chapter 1: Introduction to Agentic Apps 1 The advent of chatbots and LLM-powered applications ............................................................ 2 New ways of building apps – context engineering ..................................................................... 2 Where do we start as developers? .............................................................................................. 3 Chapter 2: Building Responsible AI Systems 5 The impossible itinerary reveals a bigger problem .................................................................... 6 First ask whether AI belongs in the system • 7 Ask five responsible AI questions • 8 Quick practice: name the primary concept • 9 Draw a boundary around what the system can do • 10 One incorrect suggestion begins to spread ............................................................................... 11 Problem 1: the AI presents an unsupported claim • 11 Problem 2: the application grants too much authority • 11 Problem 3: one error creates more errors • 12 Exercise 1: finding where the trip goes wrong • 12 The team adds safeguards before the next booking .................................................................. 13 Guardrail 1: give the AI clear instructions • 13 Guardrail 2: limit which actions are available • 14 Guardrail 3: check permissions before acting • 14 Guardrail 4: check important facts independently • 15 Guardrail 5: keep the traveler in control • 15 Exercise 2: choosing safeguards for a risky request • 16 The booking request introduces a privacy risk ........................................................................ 16 Ask for personal data only when it is needed • 16 Protect the traveler's data wherever it goes • 17 Let the traveler see and control their information • 17
Page
9
Exercise 3: protecting data and explaining the request • 18 A traveler complains: can the team explain what happened? .................................................. 18 Show what is known, uncertain, and not yet done • 18 Keep a useful record without copying secrets • 19 Reconstruct the full journey from request to outcome • 19 Before launch, the team plans for failure ................................................................................. 19 Start with the traveler and the intended task • 20 Test, release, and learn • 20 Assignment: prepare the travel system for its next booking ..................................................... 21 Follow one booking journey • 21 Submission templates • 22 What a strong safety plan shows • 23 Solution .................................................................................................................................. 23 Summary ................................................................................................................................ 23 Chapter 3: Introduction to Large Language Models (LLMs) 25 Tools of the trade .................................................................................................................... 26 Prompts .................................................................................................................................. 29 System and user messages • 30 Challenge: a recipe prompt • 31 Prompt engineering ................................................................................................................. 31 Role-based prompting • 32 Combining techniques with the CREATE framework • 32 Few-shot prompting • 33 Challenge • 33 Prompt chaining • 33 Step-by-step decomposition • 34 Challenge: decomposing an application plan • 35 Maieutic prompting • 36 Exercise: maieutic prompting • 39 Models and where to find them .............................................................................................. 40 Local models • 40 Cloud-hosted models • 42 Model parameters ................................................................................................................... 43 Exercise: model parameters • 44 Table of Contents viii
Page
10
Tokens, context windows, and costs ....................................................................................... 46 Tokens, the smallest unit of text that the model processes • 46 Exercise: calculating tokens • 46 Context window • 47 Steering model output ............................................................................................................ 48 System messages, apply behavior through an API • 49 Structured outputs, get a machine-readable response • 50 Knowledge check ..................................................................................................................... 51 Summary ................................................................................................................................. 51 Chapter 4: Building with LLMs 53 Using a software development kit (SDK) ................................................................................. 54 Chapter map • 54 Our first app ............................................................................................................................ 54 Exercise: building a simple CLI assistant • 55 Streaming ................................................................................................................................ 57 Exercise: improving responsiveness with streaming • 58 Chat messages ......................................................................................................................... 59 Achieving better results with system prompts ........................................................................ 61 Exercise: using system prompts to set the behavior of the model • 62 Caution: it's hard to shut down a model ................................................................................. 63 Translation app example • 64 Configuring the model parameters ......................................................................................... 67 Exercise: configuring the model's parameters to balance creativity and determinism in its responses • 68 Preparing the app for machine readability ............................................................................... 71 System message instructions • 72 Format options • 73 Providing a type • 73 Exercise: structured outputs • 74 Summary ................................................................................................................................ 74 Chapter 5: Enhancing LLMs with Tool Calling 75 Tool calling ............................................................................................................................. 76 Defining a tool • 78 ix Table of Contents
Page
11
Instructing the LLM on how to call the tool • 79 Handling tool calls and errors • 80 Tool calling in Ollama ............................................................................................................. 82 Defining a tool in Ollama • 83 Passing the tool schema to the LLM • 84 Handling tool calls • 85 Example: call an API ................................................................................................................ 87 Writing the function • 87 Defining the tool schema • 90 Passing the tool schema to the LLM and handling tool calls • 90 Example: query a database ...................................................................................................... 92 Writing the function querying the database • 93 Defining the tool schema • 94 Passing the product tool schema to the LLM • 95 Walking through the complete api.py application • 98 Copilot SDK and how it simplifies tool calling ....................................................................... 101 Assignment ........................................................................................................................... 102 Solution ................................................................................................................................ 103 Summary .............................................................................................................................. 103 Chapter 6: Introduction to Retrieval-Augmented Generation (RAG) 105 Retrieval-augmented generation .......................................................................................... 106 Challenge • 109 Ingestion ................................................................................................................................ 110 Ingestion step 1: chunking ...................................................................................................... 114 Ingestion step 2: embeddings ................................................................................................. 116 How to generate embeddings • 117 Query-time step 1: similarity search ....................................................................................... 118 Search metrics • 118 Cosine similarity • 119 Chunking in practice: text splitting ........................................................................................ 122 Simple text splitting • 122 Markdown text splitting • 123 Ingestion step 3: vector storage ............................................................................................. 128 In-memory vector store • 128 Table of Contents x
Page
12
Using sqlite-vector for vector storage • 134 Ingestion • 134 Chroma, a popular open-source vector database • 137 Step 1: set up the Chroma client and collection • 138 Step 2: ingest a document into the Chroma collection • 139 Step 3: retrieve from the Chroma collection • 140 Improving RAG with LLMs ..................................................................................................... 143 Reranking example • 143 Answer generation examples • 145 Contextual understanding example • 145 Handling out-of-distribution queries example • 146 Assignment ............................................................................................................................ 147 Solution ................................................................................................................................ 148 Summary .............................................................................................................................. 148 Chapter 7: GraphRAG, a Retrieval Approach for Relationship-Heavy Questions 149 Key terms .............................................................................................................................. 150 When to use GraphRAG .......................................................................................................... 151 A security operations example • 152 Graph database vs GraphRAG • 153 GraphRAG by example • 153 Retrieval flow in GraphRAG .................................................................................................... 154 Cypher in one minute • 155 Why use Cypher instead of another graph query language? • 156 Building a GraphRAG pipeline ............................................................................................... 158 Step 1: defining the schema • 158 Step 2: extracting entities and relationships • 159 Step 3: storing the graph • 164 Debugging the pipeline one checkpoint at a time • 167 Step 4: planning and running retrieval • 167 1. Converting the prompt to a query plan • 168 2. Converting the query plan to Cypher • 171 3. Running Cypher • 172 4. Returning data • 173 xi Table of Contents
Page
13
Moving to Neo4j ..................................................................................................................... 174 1. Setting up Neo4j • 174 2. Choosing a runtime and package • 175 3. Configuring the connection • 175 4. Running a query • 176 Pitfalls and best practices ....................................................................................................... 177 Assignment ........................................................................................................................... 178 Solution ................................................................................................................................ 178 Summary .............................................................................................................................. 178 Chapter 8: The ReAct Pattern 179 ReAct vocabulary ................................................................................................................... 181 What is ReAct? ....................................................................................................................... 181 Tool calling in ReAct .............................................................................................................. 182 Defining an agent with LLMs and tools ................................................................................. 184 When should you use ReAct? • 190 State and memory ................................................................................................................. 190 Handling constraints and side effects • 192 Guardrails for tool use • 192 Implementing error recovery ................................................................................................. 193 Traces ..................................................................................................................................... 193 Assignment ........................................................................................................................... 194 Solution ................................................................................................................................ 196 Summary .............................................................................................................................. 196 Chapter 9: Building an AI Assistant 197 Scope and what you will build ............................................................................................. 200 Use case: IT support assistant .............................................................................................. 200 Architectural overview .......................................................................................................... 201 Application architecture • 202 Assistant architecture • 202 Architecture decision checkpoints • 205 Suggested project layout ....................................................................................................... 205 Folder walkthrough • 207 Exercise 1: minimal terminal assistant .................................................................................. 207 Table of Contents xii
Page
14
Common mistakes in exercise 1 • 209 Conversation memory ........................................................................................................... 209 Memory design for this use case • 211 What to remember vs. what to summarize • 212 Exercise 2: adding short-term memory .................................................................................. 212 Common mistakes in exercise 2 • 214 Tool orchestration ................................................................................................................. 214 End-to-end walkthrough: one request through the system • 214 Tool selection • 216 Sequencing • 217 Error handling • 217 Pattern comparison • 218 Choosing patterns with a simple decision rule • 219 Pattern as code examples • 219 Minimal error-handling rules • 221 Exercise 3: adding planning and orchestration ...................................................................... 222 Common mistakes in exercise 3 • 223 Hallucination mitigation (practical level) ............................................................................. 224 UX considerations for AI latency ........................................................................................... 225 Perceived responsiveness strategies • 225 Progressive disclosure for longer operations • 226 Exercise solutions .................................................................................................................. 226 Design checklist before the assignment ................................................................................ 226 Assignment ........................................................................................................................... 227 How to self-review your result • 227 Solution ................................................................................................................................ 228 Summary .............................................................................................................................. 228 Chapter 10: Agent Architecture: From AI Assistants to Agents 229 Architecture decisions ........................................................................................................... 230 How agent architecture connects to ReAct ............................................................................. 231 Assistant vs. agent .................................................................................................................. 231 Core components of an agent ................................................................................................ 235 The pure agent: LLM-driven control ..................................................................................... 237 1. Identifying a goal and a plan • 238 xiii Table of Contents
Page
15
2. Action loop with observation and completion check • 239 3. Evaluation and replanning • 240 Exercise 1: running and tweaking 00-agent.py • 241 Where the pure agent breaks down ....................................................................................... 241 The risk spectrum: choosing the right architecture ............................................................... 242 The workflow agent: predictable execution .......................................................................... 243 How the workflow engine works • 244 Assignment ........................................................................................................................... 246 Summary .............................................................................................................................. 248 Chapter 11: From Agent Architecture to Planning Systems 249 Planning systems for agents ................................................................................................... 251 Core components of a planning system ................................................................................. 253 State: what flows through the loop ....................................................................................... 255 Planning system design patterns ........................................................................................... 256 Implementing a planner • 257 Executor ................................................................................................................................ 258 When the executor should stop • 260 Monitor ................................................................................................................................. 262 Paper exercise: pizza ordering domain .................................................................................. 265 Exercise: implementing a simple planning system with a planner, executor, and monitor ... 267 Running the baseline first • 268 Then adding one extension • 268 Planning system design patterns in depth ............................................................................. 268 Hierarchical task networks (HTN) • 269 Reactive planning • 270 Constraint-based planning • 271 Iterative planning • 273 Dynamic replanning ............................................................................................................. 274 Exercise: dynamic replanning • 276 Constraint-aware planning ................................................................................................... 278 How constraints reach the planner • 280 Exercise: constraint-aware • 282 Planner-executor-monitor coordination ............................................................................... 283 Exercise: contract preservation • 285 Table of Contents xiv
Page
16
Measuring planning quality and execution success .............................................................. 286 Exercise: measurement feedback loop • 287 Assignment ........................................................................................................................... 288 Solution • 289 What we learned building the assignment • 289 Production readiness: what this chapter covers vs. what's missing ...................................... 290 Summary .............................................................................................................................. 291 Chapter 12: Agent Orchestration and Autonomy 293 A control layer roadmap ........................................................................................................ 294 Why orchestration matters ................................................................................................... 296 Orchestration patterns .......................................................................................................... 298 Sequential orchestration • 298 Parallel orchestration • 300 Routed orchestration • 302 Exercise 1: patterns ................................................................................................................ 305 Step 1: running sequential orchestration • 306 Step 2: running parallel orchestration • 307 Step 3: running routed orchestration • 307 Step 4: comparing behavior in the web workbench • 307 Mini change to try • 309 Autonomy levels .................................................................................................................... 310 Trying the same authority boundary in other systems • 313 Exercise 2: autonomy levels and approval gates ..................................................................... 314 Checkpoints and recovery ...................................................................................................... 317 Safe execution boundaries ..................................................................................................... 321 Observability for autonomous decisions ............................................................................... 323 Exercise 3: adding checkpointing .......................................................................................... 325 Exercise 4: adding an approval gate ....................................................................................... 325 Assignment ........................................................................................................................... 326 Solution ................................................................................................................................ 327 Summary .............................................................................................................................. 327 Chapter 13: Multi-Agent Collaboration and Communication 329 From one agent to a team ....................................................................................................... 331 xv Table of Contents
Page
17
Defining the mission ............................................................................................................. 332 Building the team .................................................................................................................. 335 Exercise 1: choosing the travel team ...................................................................................... 338 Establishing the language ..................................................................................................... 339 Choosing the control style ..................................................................................................... 349 Handling disagreement .......................................................................................................... 351 Exercise 2: adding a supervisor ............................................................................................. 355 Exercise 3: adding a conflict rule ........................................................................................... 357 Surviving failure ................................................................................................................... 357 Exercise 4: checkpoint and resume ........................................................................................ 359 Making it explainable ........................................................................................................... 362 Proving it works .................................................................................................................... 363 Solution ................................................................................................................................ 364 Summary .............................................................................................................................. 364 Chapter 14: Multi-Agent Patterns 367 Where you are and the next problem ..................................................................................... 368 Understanding the core patterns and families ...................................................................... 369 Quick start: one working example ........................................................................................ 373 Start here (15 minutes) • 373 Tiny glossary • 374 Scoring card: evaluating each pattern on your problem • 375 The supervisor-worker pattern: delegation and task routing ................................................ 375 When to use it • 376 How it works in practice • 376 Final thoughts • 377 Exercise 1: building the supervisor-worker baseline • 377 The debate pattern: competing viewpoints and judgments .................................................. 378 When to use it • 378 How it works in practice • 378 Final thoughts • 379 Exercise 2: running a debate on one travel trade-off • 380 The swarm pattern: parallel problem solving and consensus ................................................ 380 When to use it • 380 How it works in practice • 381 Table of Contents xvi
Page
18
Final thoughts • 382 Exercise 3: comparing parallel plans with swarm • 382 The committee pattern: shared discussion and voting .......................................................... 382 When to use it • 383 How it works in practice • 383 Final thoughts • 385 Exercise 4: running a committee vote with dissent • 385 The peer-to-peer pattern: negotiation and direct exchange .................................................. 385 When to use it • 386 How it works in practice • 386 Final thoughts • 388 Exercise 5: negotiating a travel decision peer-to-peer • 389 Choosing your pattern: a decision framework ...................................................................... 389 Measuring these: latency, cost, and output quality ............................................................... 393 Metrics you should track • 393 Knowing when patterns break: scaling limits and failure modes .......................................... 394 Scaling limits • 394 Failure modes and how to avoid them • 395 When to pick which pattern .................................................................................................. 396 Building in safety by structure: design patterns that enforce policy • 397 Assignment ........................................................................................................................... 398 Solution ................................................................................................................................ 399 Summary .............................................................................................................................. 399 Chapter 15: Agent Design Patterns and Anti-Patterns 401 Where you are and the next design problem ......................................................................... 402 Choosing the right pattern from the problem shape ............................................................. 403 Pattern selection matrix • 404 Exercise 1: pattern selection .................................................................................................. 405 Understanding the core design patterns .............................................................................. 406 Pattern 1: orchestrator + specialists • 406 Pattern 2: planner + executor loop (bounded) • 407 Pattern 3: retrieval-first decisioning • 407 Understanding the anti-patterns that cause real failures ..................................................... 408 Anti-pattern 1: unbounded autonomy • 408 xvii Table of Contents
Page
19
Anti-pattern 2: tool roulette • 408 Anti-pattern 3: hidden state coupling • 409 Anti-pattern 4: evaluation last • 409 Exercise 2: predicting failure ................................................................................................. 410 Designing guardrails for safety and reliability ....................................................................... 410 Guardrail 1: input and policy validation • 411 Guardrail 2: action confirmation tiers • 411 Guardrail 3: fallback and containment • 412 Guardrail 4: tracing everything important • 412 Exercise 3: guardrail design ................................................................................................... 412 How to refactor a brittle design using patterns, anti-patterns, and guardrails ....................... 413 Use case • 413 Let's refactor • 414 Carrying design artifacts into Chapter 16's evaluation ........................................................... 415 Carrying design artifacts into the final chapters ................................................................... 416 Assignment ............................................................................................................................ 417 Summary .............................................................................................................................. 418 Chapter 16: Testing LLMs and Agents with Evaluation Frameworks 419 How do I even evaluate, and how is it different from normal unit testing? ............................ 420 What changes in LLM and agent testing? • 421 Exercise 1: converting a unit test into an evaluation test ....................................................... 422 Defining metrics: what does good look like ........................................................................... 423 Tools and frameworks: how to automate evaluation ............................................................. 423 Evaluation workflow: how to put it all together .................................................................... 425 Dataset: how to create a representative set of prompts and responses • 425 Exercise 2: creating a dataset of prompts and expected responses ......................................... 426 Building your first evaluation ................................................................................................ 428 Evaluation with Promptfoo • 428 Description • 428 Prompts • 429 Model provider and configuration • 429 Tests • 429 Running the evaluation • 431 From scores to pass/fail thresholds • 432 Table of Contents xviii
Page
20
Exercise 3: running the evaluation and inspecting the results ............................................... 434 What the baseline gives you .................................................................................................. 434 Testing agents: tool use, state, and control flow .................................................................... 435 Agent metrics for multi-step behavior • 435 Testing multi-step agent behavior • 436 Exercise 4: inspecting multi-step agent behavior .................................................................. 437 Assignment ........................................................................................................................... 438 Solution ................................................................................................................................ 439 Summary .............................................................................................................................. 439 Chapter 17: Building an Agent Harness 441 What is agent harness engineering? ...................................................................................... 442 Starting with an agent without a harness ............................................................................. 443 What happens without a harness? ........................................................................................ 445 Building the harness one boundary at a time ........................................................................ 446 Exercise 1: adding and observing a step limit ......................................................................... 448 Separating the runtime responsibilities ................................................................................ 449 Adding policy, deadlines, and recovery boundaries ................................................................ 451 Exercise 2: adding an approval boundary .............................................................................. 457 Making agent runs observable .............................................................................................. 458 Exercise 3: recording the denied trajectory ............................................................................ 459 Designing the evaluation harness ........................................................................................ 460 Evaluating agent outcomes and trajectories .......................................................................... 462 Evaluating the harness itself ................................................................................................. 463 Exercise 4: testing the harness without Ollama .................................................................... 464 Combining deterministic tests with live scenarios ................................................................ 464 Exercise 5: deciding whether to deploy a candidate agent ..................................................... 465 Comparing agent versions before updating the model .......................................................... 466 Do we update the agent with this new model? • 466 Keep evaluating the agent in production ............................................................................... 467 Assignment ........................................................................................................................... 468 Checkpoint 1: bounding one run • 469 Checkpoint 2: guarding and recording actions • 469 Checkpoint 3: loading scenario contracts • 469 Checkpoint 4: evaluating isolated runs • 469 xix Table of Contents
The above is a preview of the first 20 pages. Register to read the complete e-book.
AI Reading Assistant
Whole-book reading guide from stratified index samples; jump to passages in the text
AI guide
【One-Line Pitch】
A hands-on field guide for web developers who want to move past "chat with an LLM" demos and build autonomous, tool-using agents that actually ship. Read it if you already write web apps and want a practical path from prompt engineering to orchestrated multi-agent systems.
【Book Arc】
- **Opening (~0%–10%)**: Frames the shift from text generation to goal-driven agentic systems, then sets up the toolchain — hosted playgrounds like GitHub Models and local runtimes like Ollama — so you can experiment before committing to an architecture.
- **Early (~10%–30%)**: Builds the prompting and responsibility foundation: steering output with parameters, tokens/context/cost trade-offs, decomposition and self-correction techniques (e.g., maieutic prompting), and a chapter on responsible AI that asks whether AI belongs in a given task at all.
- **Middle (~30%–55%)**: The core capability layer — tool calling (schemas, execution loops, error handling, API and database examples) followed by RAG: chunking strategies, embeddings, vector storage, and similarity search.
- **Late (~55%–85%)**: Moves from single assistants to agent systems — reasoning patterns like ReAct, agent architectures, planning, orchestration, multi-agent collaboration, and MCP as the interoperability layer.
- **Ending (~85%–100%)**: Production concerns: testing and evaluating LLMs and agents, building an "agent harness" of context, tools, guardrails, and infrastructure, plus runtime checkpoints and failure-pattern detection before deploying a candidate.
【Key Takeaways】
- **Agents are goal-driven systems, not chat wrappers** (Opening): the book's framing is that reasoning, acting, and collaborating are the product, and everything else is scaffolding.
- **Tool calling is the hinge between LLM and real world** (Middle): define a function, expose its schema, intercept the model's call, execute, and feed results back — the same loop transfers across SDKs like Ollama and Copilot SDK.
- **RAG is a pipeline with tunable trade-offs, not a checkbox** (Middle): chunking choices (fixed-size, semantic, overlap, dynamic, hierarchical) directly determine retrieval quality, and there is no universal right chunk size.
- **Prompting is engineering, not magic** (Early): decomposition, self-reflection, and parameter tuning (temperature, top-p) shape reliability, and the book shows models can still ignore instructions — so test across models.
- **Responsibility scales with consequence** (Early): the book argues for asking "does this task need AI at all?" and matching safeguards — confirmation, permission, human review — to how reversible and sensitive an action is.
- **MCP is positioned as the interoperability layer** (Late): it connects agents to tools, resources, and capabilities so systems stay composable rather than bespoke.
- **Evaluation and harnesses are what make agents production-ready** (Ending): testing, runtime checkpoints, and guardrails are treated as first-class engineering, not afterthoughts.
- **Failure detection is a design concern** (Ending): the book discusses determining whether a failure is a pattern and detecting when that pattern needs attention.
【Reading Tips】
- Deep-read the tool calling and RAG chapters (~30%–55%); they are the load-bearing skills everything later assumes. Skim the environment setup and playground sections if you already have a working LLM stack.
- Treat the responsible AI chapter as a decision framework, not compliance boilerplate — its "does this need AI?" question will save you architecture time later.
- Work the assignments and checkpoints rather than reading passively; the book's value is in the round-trip loops you build yourself.
- When you reach multi-agent and orchestration material, keep asking "could one agent with better tools do this?" — the book's patterns are options, not mandates.
- Note the model-portability warnings (e.g., instruction-following differences between models) and test your prompts against at least two models before locking in.
【Coverage Limits】
This guide is synthesized from stratified excerpts and the book's front matter; specific chapter-level detail on planning systems, multi-agent orchestration, and the agent harness is thinner in the available material than the tool calling and RAG sections, so those later stages are mapped at a higher level.
Passage locations
Excerpt 1
y, intelligent web experiences powered by autonomous agents. What you will learn Understand agentic thinking and responsible AI principles Integrate LLMs int...
View in text
Excerpt 2
annot prevent unnecessary collection of passport details. A confirmation step can prevent an unwanted purchase, but it cannot make an inaccurate price truthf...
View in text
Excerpt 3
etting a limit on the number of output tokens the model can generate, but you also need to be mindful of the number of input tokens you are sending, as that...
View in text
Excerpt 4
pproach involves breaking the text into chunks based on its semantic structure, such as paragraphs, sections, or sentences. This can be more effective in cap...
View in text
Recommended for You
{{#thumbnailUrl}}
{{/thumbnailUrl}}
{{^thumbnailUrl}}
{{/thumbnailUrl}}
Loading recommended books...
Failed to load, please try again later
Tip the Site
Scan the WeChat Pay or Alipay code to tip. No login required.
WeChat Pay
Alipay