Share E-Book

Multimodal, Real-Time AI Agent Systems (First Early Release) (Heiko Hotz, Sokratis Kartakis)(Z-Library)

Author

,

Rating 3.5 AI initial rating

Log in to rate

AI
Language English

As users increasingly expect AI to interact as naturally as humans, engineers must move beyond static prompts to building systems that perceive, reason, and act instantly, mimicking human interaction more closely. This book takes you from agent fundamentals to architecting advanced, bidirectional, multimodal, multi-agent systems, focusing on the difficult leap from proof of concept to production. You will start by building agents directly with foundation models to understand core components and then master modern agent frameworks that simplify and scale implementation. Written by industry practitioners, this book connects agentic concepts with cutting-edge standards and protocols of interoperability. You will learn to build enterprise-grade agent platforms that enforce scalable AgentOps, rigorous evaluation, and extreme security measures required for live streaming interactions.

Format EPUB
Size 14.7 MB

AI Guide

AI Reading Assistant

Whole-book reading guide from stratified index samples; jump to passages in the text

Full assistant
AI guide
【One-Line Pitch】 A practitioner's field guide for engineers who need to move AI agents from clever demos to live, multimodal, multi-agent systems that hold up in production. Read it if you're building real-time conversational or agentic products and care about architecture, interoperability, and operations—not just prompt tricks. 【Book Arc】 - **Opening (~0%–10%)**: Sets the ambition—agents that perceive, reason, and act in real time—and frames the whole journey from foundation-model basics to enterprise-grade platforms. Establishes why static prompting is no longer enough. - **Early (~10%–35%)**: Builds the conceptual floor: a five-stage evolution of agents (classic deliberative/reactive/hybrid designs → foundation models → RAG → tool-using agents → multi-agent orchestration), a working definition of an agent, and the perception–reasoning–action loop. - **Middle (~35%–55%)**: Gets concrete on the mechanics—how tools are declared (function declarations, docstrings, OpenAPI/JSON, SDK-based definitions) and how the agent execution loop orchestrates model calls and tool results. This is the "build it yourself before using a framework" stage. - **Late (~55%–85%)**: Moves into real-time multimodal interaction and agentic architecture: live APIs, video/tools/system instructions, agent frameworks, multi-agent collaboration, and the interoperability protocols (A2A, MCP) that let agents and tools talk to each other. - **Ending (~85%–100%)**: Productionization—an integrated reference architecture, async and long-running tasks, UX design, then AgentOps, evaluation/testing, tool management at scale, governance, security/privacy/responsible AI, and deployment/scaling. 【Key Takeaways】 - **Agents are systems, not prompts** (Early): The book defines an agent as an autonomous, foundation-model-powered system that perceives, decides, and acts via tools toward a goal—shifting attention from clever prompts to the full sense–think–act cycle. - **History still pays off** (Early): Classic deliberative, reactive, and hybrid agent architectures supply the blueprints; LLMs simply drop a far stronger reasoning engine into them. Understanding this makes modern design choices legible rather than magical. - **The FM transition trades transparency for power** (Early): Compared with rule-based classical agents, FM-powered agents reason probabilistically, plan dynamically, and pick tools from natural-language descriptions—but become opaque boxes that are harder to debug and explain. - **Tool declarations are the real interface** (Middle): An agent is only as capable as its tools, and the model learns tools through descriptions—name, purpose, inputs, outputs, and worked examples. Well-written declarations (few-shot style) are a design skill, not boilerplate. - **Standardize before you scale** (Middle): Moving from ad-hoc docstrings to formal OpenAPI/JSON definitions buys predictability, robustness, and integration—and sets up the later protocol chapters (MCP for tools, A2A for agent-to-agent). - **The execution loop is the heart of orchestration** (Middle): The back-and-forth of query → tool call → result → model response is the pattern every framework abstracts; building it manually first makes frameworks understandable rather than mysterious. - **Real-time and multimodal raise the bar** (Late): Live, bidirectional interaction with video, tools, and system instructions demands architecture decisions that differ sharply from request/response chatbots. - **Production is an operations problem** (Ending): AgentOps, rigorous evaluation, tool management at scale, governance, and security are treated as first-class engineering concerns—the gap between PoC and live system lives here. 【Reading Tips】 - Deep-read Chapters 1 and the tool/execution-loop material; that foundation makes the framework and protocol chapters feel like shortcuts rather than new material. - Skim the historical evolution if you already know agent taxonomy—but keep the "then vs. now" comparison table in mind as a design checklist. - Treat the protocol chapters (A2A, MCP) as reference material to revisit when you actually wire systems together; don't try to memorize specs on first pass. - Note that this is an Early Release: some chapters are marked unavailable in the excerpts, so expect gaps and check the final edition for production chapters. - Keep a running list of your own tool declarations and evaluation criteria as you read—the book rewards readers who apply it to a live project. 【Coverage Limits】 The excerpts cover the opening, Chapter 1's agent foundations and tool mechanics, and the table of contents; most later chapters (multimodality, real-time APIs, frameworks, protocols, production) are listed but not excerpted, so this guide's treatment of those stages is structural rather than detailed.

Passage locations

Excerpt 1
are also available for most titles ( https://oreilly.com ). For more information, contact our corporate/institutional sales department: 800-998-9938 or corpo...
View in text
Excerpt 2
y principles of reasoning and planning we’ve just discussed. Stage 1: The Foundation Models and Prompts In the Generative AI era, it all started with the Lar...
View in text
Excerpt 3
the old systems for a massive gain in power and flexibility. Note We’ve covered the important history of legacy agents, but to keep things clear from this po...
View in text
Excerpt 4
in the functionDeclarations list to complete your Math tool. While writing JSON files by hand is a great way to understand the structure of a tool, in a real...
View in text

Recommended for You

Loading recommended books...
Failed to load, please try again later

Tip the Site

Scan the WeChat Pay or Alipay code to tip. No login required.

WeChat Pay
Alipay
← Back to List