Share E-Book

AI Engineering in Practice (MEAP) (Richard Davies, Rafael Fischer)(Z-Library)

Author

,

AI
Language English

Write, refine, organize, and optimize AI prompts that generate relevant and useful text and images! Generative AI models such as ChatGPT, Stable Diffusion, and Gemini can produce amazingly “human-like” news articles, document summaries, images, computer code, and more—if you know how to write effective prompts. This book will teach you the prompt design and authoring skills you need to get useful and relevant responses from AI models, along with advanced prompting techniques for Retrieval Augmented Generation (RAG), building autonomous agents, and data privacy. AI Engineering in Practice teaches you how to: Design prompts that generate accurate and readable responses from LLMs Mitigate hallucinations in LLM output Domain-aware content generation using RAG How AI model design affects your prompts Evaluate, optimize, and organize your prompts Prompt engineering is the discipline of writing instructions for AI models to generate relevant, accurate, and usable completions. AI Engineering in Practice shows you how to engineer prompts that ensure the outputs of LLMs and other generative AI models exactly match your requirements. You’ll learn how to structure your objectives, take advantage of contextual details, and even pick the right model for your task. about the reader No special skills with AI or machine learning required. Code examples are in Python.

Format PDF
Size 1.7 MB
11
Views
(First 20 pages)

Registered users can read the full content for free

Register as a Gaohf Library member to read the complete e-book online for free and enjoy a better reading experience.

Page 1
(This page has no text content)
Page 2
(This page has no text content)
Page 3
AI Engineering in Practice 1. welcome 2. 1_AI_Engineering_-_The_Blueprint 3. 2_Foundation_Models:_Language_&_Embedding 4. 3_Prompt_Design:_Structural_Elements 5. 4_Prompt_Design:_Linguistic_Elements 6. 5_Prompt_Patterns 7. 6_Prompt_Templates
Page 4
welcome Dear Reader, Thank you for purchasing the MEAP for Principles of AI Engineering in Practice. This book is designed to provide you with a comprehensive understanding of prompt engineering, a rapidly evolving field that enables practitioners to harness the power of large language models (LLMs) for a wide range of applications. To get the most out of this book, you should have a general understanding of computer usage and be familiar with the concept of artificial intelligence. Previous programming experience is beneficial but not required, as the book will guide you through the process of designing effective prompts and templates to elicit desired outputs from LLMs. As the CTO of Vance, an artificial intelligence startup in the regulatory compliance space, and with over 6 years of industry experience, I have developed a deep passion for Artificial Intelligence. I have successfully applied techniques (including Large Language Models and Prompt Engineering) to create cutting-edge products, such as a real-time semantic segmentation system for ultrasound images, a thermal activity detection algorithm for monitoring elderly patients, and a machine translation platform capable of translating across 60+ languages. Throughout this book, you will learn the fundamentals of prompt engineering, including structural and linguistic elements, patterns, templates, and advanced techniques (Chain-of-Thought, Tree of Thought, Retrieval Augmented Generation). You will also explore real-world applications, such as building chatbots, extracting information from documents, and creating autonomous agents. Additionally, the book covers prompt evaluation, optimization, security, and best practices to help you effectively manage and scale your prompt engineering projects.
Page 5
One of the unique aspects of this book is its focus on autonomous agents and automated prompt optimization. You will learn how to design and implement autonomous agents that can perform tasks and make decisions based on prompts and context, opening up new possibilities for AI-driven applications. Additionally, the book explores cutting-edge techniques for automating the optimization of prompts, enabling you to create high-performing prompts with minimal manual effort. I am excited to share my knowledge and experience with you through this book. By the end of it, you will have gained the skills and confidence to apply prompt engineering techniques to your own projects and contribute to the rapidly evolving field of artificial intelligence. Please feel free to share your thoughts, questions, and feedback in the liveBook Discussion forum. Your input is invaluable in shaping the final version of the book and ensuring that it meets your needs and expectations. Thank you again for your interest and for purchasing the MEAP for AI Engineering in Practice. I look forward to embarking on this learning journey with you! Best regards —Richard Davies In this book welcome 1 AI Engineering - The Blueprint 2 Foundation Models: Language & Embedding 3 Prompt Design: Structural Elements 4 Prompt Design: Linguistic Elements 5 Prompt Patterns 6 Prompt Templates
Page 6
1 AI Engineering - The Blueprint This chapter covers Distinguishing AI Engineering from Prompt Engineering. Evaluating when simple prompting fails and engineering is needed for scale, integration, or managing risk. Understanding the five architectural layers—routing, RAG, prompts, agents, and infrastructure. Following a customer query through a production pipeline to observe routing, retrieval, and validation in action. Diagnosing failure symptoms, like hallucinations or cost overruns. In February 2024, Air Canada's AI-powered customer service chatbot told a passenger he could request a bereavement fare refund after travel, a policy that did not exist. The airline was ordered by a Canadian tribunal to honor the misleading guidance and issued a partial refund. The incident was not caused by the model itself but by the absence of disciplined AI engineering practices. Developers had focused on conversational design and prompt tuning, and the chatbot worked smoothly in demonstrations. But production AI requires architecture, not just better prompts. Reliable systems demand automated evaluation pipelines, input validation, hallucination detection, and continuous monitoring. They must also balance competing constraints: response quality, latency, cost per query, and throughput at scale. Now consider Klarna's AI assistant: launched in early 2024, it handled 2.3 million customer conversations in its first month, work equivalent to about 700 full-time agents, with customer satisfaction on par with human representatives. The difference lies in systematic AI engineering: a controllable, routed agent architecture optimizing cost and latency; retrieval systems grounding responses in Klarna's product and policy databases; multiple validation layers reducing error rates; and operational monitoring through continuous evaluation ensuring consistent quality. This chapter reveals the blueprint underlying systems like Klarna's. You will
Page 7
follow a customer query through a complete production pipeline, observing each engineering layer and the failure modes it prevents. These systems succeed not by replacing software engineering fundamentals, robust error handling, testing, logging, and modularity, but by extending them with AI- specific patterns: retrieval architectures, output validation, and model routing strategies. You will see why ad-hoc prompting collapses at scale, and how systematic architecture, built on solid engineering foundations, establishes reliability. 1.1 What is AI Engineering? AI Engineering is Software Engineering that incorporates modern AI techniques, Language Models, Vector Databases, Embeddings, to solve problems involving unstructured data like text, images, and audio. It requires the same discipline as traditional Software Engineering: scalable architecture, systematic testing, robust error handling, and operational monitoring. The distinction lies not in abandoning software engineering fundamentals, but in extending them with AI-specific patterns for problems where traditional approaches fall short. The capability to process, classify, and extract insights from unstructured data enables solving problems that were previously intractable or required extensive manual effort, making solutions both more capable and more efficient. AI Engineering builds production systems with the modern AI stack, applying architectural patterns that decompose complexity, integration strategies that connect AI to existing systems, validation frameworks that ensure consistent quality, and operational practices that make systems observable and maintainable. Prompt Engineering provides an interface to Language Models, the techniques for effective communication with AI systems. AI Engineering builds production systems around those interfaces: architectures for reliability, validation for quality control, routing for cost optimization, and operational practices for scalability. Where Prompt Engineering teaches you to communicate with models, AI Engineering teaches you to build production systems with the modern AI stack.
Page 8
1.1.1 From Prompts to Production Systems Large Language Models (GPT, Claude, Gemini, Llama, Mistral) have made AI accessible, but accessibility created a critical misconception: that Language Models can solve any problem because prompting ChatGPT works for single examples. Business leaders interact with ChatGPT, see it handle their test case successfully, and assume deployment is straightforward. This demo-to-production gap becomes apparent at scale, serving thousands of customers reveals challenges invisible in individual tests. Production systems require software architecture to ensure consistent quality, manage context window limitations, optimize costs, and handle edge cases that rarely appear in demos but occur daily at scale. Figure 1.1 The Demo-to-Production Gap
Page 9
Consider a simple example that illustrates why even basic prompt use requires systematic thinking, and why systematic prompting itself requires engineering discipline for production use. You are using Language Models to generate product descriptions. Even this straightforward task reveals the difference between ad-hoc prompting and systematic Prompt Engineering. Ad-hoc Approach Write a product description for wireless headphones.
Page 10
This ad-hoc prompt produces generic output varying wildly in quality, style, and length. Systematic Approach # Product Details Product: {product_name} Features: {features} Audience: {audience} # Instructions You are an expert e-commerce copywriter. Write a product description for the `{product_name}` using the provided features and targeting the specified audience. Use an enthusiastic but credible tone. # Output Format [Hook highlighting the main benefit] [Key features with customer benefits] [Relatable use case scenario] [Call to action] This systematic prompt produces consistent, on-brand descriptions reliably across thousands of products. The structured format ensures output meets your requirements. Now scale this to 10,000 products daily. Costs must stay under $500/month. Quality must remain consistent. Outputs must populate your database automatically. The template needs parameterization so it can be called programmatically. Outputs need structured formatting for database insertion. You need validation to catch quality regressions. You need monitoring to track costs and performance. You need software architecture to overcome context window limits when product catalogs contain thousands of items.
Page 11
This is where Prompting becomes engineering: structured schemas enabling database integration, parameterized templates for systematic invocation, validation frameworks catching quality issues, cost optimization through model selection, routing strategies to manage context limits, and monitoring ensuring sustained performance. This book's structure reflects that integration Part 1: Prompt Engineering (Chapters 3-10) establishes the human- AI interface layer, structural elements organizing prompts, linguistic techniques controlling precision, reusable patterns, and parameterized templates Part 2: AI Engineering (Chapters 11-19) advances into systems architecture, chaining prompts into workflows, routing requests intelligently, grounding responses through RAG, deploying autonomous agents, implementing operational practices At this point, you might ask: if AI Engineering requires more investment than ad-hoc prompting, when is it worth it? And how does it compare to other approaches like rule-based systems or classical machine learning? Table 1.1 maps these alternatives, showing when each approach makes sense and where AI Engineering provides the most value. Table 1.1 AI Engineering vs. Alternative Approaches Approach How It Works Strengths Limitations Best For Rule-Based Chatbots Manual decision trees Predictable, controlled All scenarios predefined, brittle Simple workflows (< 20 paths) Classical ML Train on labeled data Domain- optimized Large datasets required, retraining needed Stable problems with abundant data Ad-Hoc Trial-and- Quick Inconsistent, Personal
Page 12
Prompting error prototypes, low investment no error handling, poor scalability productivity, exploration AI Engineering Systematic architecture Reliable at scale, maintainable, cost- optimized Higher initial investment requires skills Production apps, enterprise systems 1.1.2 When You Need AI Engineering vs. Simple Prompts Simple prompting works for personal productivity, content drafts, brainstorming, where humans review outputs, failures have minimal consequences, and quality inconsistency is acceptable. You need AI Engineering when: Integration exists: Outputs feed databases, trigger workflows, call APIs requiring structured formats Quality must be consistent: Serving thousands of users expecting reliable results Failures have consequences: Customer-facing systems, financial decisions, legal/medical contexts Cost matters at scale: Hundreds or thousands of daily requests where unoptimized prompting creates unsustainable bills Security threats exist: User-controlled inputs risk prompt injection, jailbreaking, data extraction But does AI Engineering really deliver in practice? Let's examine three organizations that discovered the difference between Prompt Engineering and AI Engineering, and the concrete results that followed. 1.2 Why AI Engineering Delivers Results AI Engineering principles may sound abstract until we see them transform real business operations. The following three case studies, from customer
Page 13
service, legal document analysis, and manufacturing operations, illustrate how systematic architecture delivers improvements that ad-hoc prompting alone cannot achieve. 1.2.1 Customer Support at Scale A mid-sized e-commerce company processing roughly 50,000 monthly tickets faced significant challenges: a 20-person support team, average response times of 48 hours, and customer satisfaction scores around 65%. Their first attempt at solving the problem with Artificial Intelligence techniques was purely prompt based, deploying a pre-trained Large Language Model (LLM) with “You are a helpful customer service agent”. The result, agents still spent as much time verifying responses as they saved, the Language Model generated hallucinated policies, lacked context awareness, and produced inconsistent brand voice. In effect, the model did not deliver meaningful improvement. The engineering approach drew from production systems such as the deployment by Klarna, which reported handling two-thirds of all chats in its first month, and enterprise agents such as FinAI and the routing model at Zendesk. The architecture comprised five layers: A Prompt routing Layer that determined whether a query should go to a lighter, lower-cost model or escalate to a stronger model (e.g., GPT-5) based on complexity. Zendesk’s benchmark-based multi-model architecture and the academic RouteLLM work document typical cost reductions of ~2× when prompt routing is applied. A Retrieval Augmented Generation (RAG) layer that indexed the company’s policies and specifications to provide context and grounding. Systems like Intercom Fin and Zendesk’s AI suite underscore the importance of retrieval in reducing hallucinations. Structured Prompt Engineering, enforcing brand voice and requiring the model to produce citations for all claims. This is aligned with documented production practices of structured outputs and “groundedness” checks.
Page 14
A validation layer that inspected model outputs for policy compliance, hallucination detection, and confidence scoring. For example, DoorDash published a guardrail system for LLMs with “auto-escalation” when confidence falls below threshold. An integration layer that automatically dispatches the validated response to the ticketing platform, tracking agent interventions and escalating when needed. In production, such architecture can yield results like the Klarna example, resolution times reduced from minutes to ~2 minutes per chat, and cost savings equivalent to several hundred full-time staff. In this hypothetical 50,000-ticket scenario the effects might include: 90% accuracy (up from 65%), average response time under 2 hours (versus 48 hours), 40% of tickets fully automated, 60% cost reduction, and customer satisfaction rising to 89%. Routing alone, by matching query complexity to model capability, can cut usage of large models and achieve large savings (in academic studies, ~40 % reduction in large-model calls). The RAG layer materially reduced hallucinations by grounding in policy documents, the validation layer prevented flawed responses reaching customers. 1.2.2 Document Intelligence in Legal Services A regional law firm was spending approximately 40 attorney-hours per contract to analyze terms, identify risks, and check compliance, translating to ~$10,000 per contract (at $250/hour). Their initial experiment used a general- purpose LLM via prompt: “Analyze this contract”. The result was missed critical clauses, hallucinated obligations, produced unstructured outputs, and lacked citations. The output was unusable for client work. Informed by deployments like A&O Shearman with its Analyze module (publicly claiming 95 % accuracy in certain workflows) and vendor systems such as Kira Systems and Evisort (which document structured-output pipelines and significant time savings), the firm adopted a structured engineering pipeline: Document chunking split contracts into logical sections (clauses, schedules, exhibits).
Page 15
Specialized prompts executed individual tasks, obligations extraction returned JSON output, risk analysis leveraged Chain of Thought (CoT) prompts; compliance checking used few-shot examples. A prompt-chaining pipeline sequenced the tasks: extract parties, obligations, risks, compliance and finally a summary. Human-in-the-Loop review: all outputs were reviewed by attorneys, and corrections fed back into Prompt Templates and extraction logic as training examples. With this engineering scaffold, they achieved, processing time reduced to ~2 hours per contract (from ~40), ~95% accuracy on extraction tasks, zero client-facing errors (because all work remained under review), ~$150,000 annual savings, and a 70% reduction in client charges. The key mechanisms, chaining tasks simplified overwhelming documents into discrete subtasks; structured output enabled systematic verification; human review maintained legal validity while capturing dramatic time savings. 1.2.3 Workflow Automation in Operations A manufacturing facility receiving handwritten inspection reports (2-3 days manual entry, error rate 8-12%) sought to automate the process. Their first attempt at automation used an Large Multi-Modal Model with Image support. The outputs contained OCR errors, inconsistent extraction, unstructured responses, and clerks spent as much time on correction as they did on manual transcription. Borrowing from successful document-automation deployments such as FAMIQ (Argentina) integrated via ABBYY into SAP for manufacturing material traceability and case studies from Clay Lacy Aviation showing ~90% reduction in manual intake errors, the facility built a robust pipeline: A Large Multi-Model Model component (e.g., GPT-4o vision or equivalent) with an explicit JSON schema defined for each field (e.g. inspection-item, pass/fail, measurement, note, date, inspector). Function calling / structured generation enforced schema adherence (type checks, valid ranges, enumeration constraints). A multi-layer validation system: schema compliance, range checking
Page 16
(e.g., measurement outside tolerance flagged), cross-field logic (e.g., “if item = X then field Y required”), and completeness checks. Retry logic: if validation fails, system records the failure reason, re-runs the extraction (often with different images or prompt variations), and logs the result. Integration via API: validated JSON automatically populates the quality-tracking system, triggering dashboards and alerts in real-time instead of waiting 2-3 days. This resulted in near real-time data entry (rather than 2-3 days delay), at roughly 98% accuracy (higher than manual’s ~92%), approximately 3 hours/week of manual correction saved, immediate visibility into quality issues, and zero integration errors. The structural factors, using schemas steered the LMM text extraction from images into predictable formats; validation and retry handled handwritten form variability, database/API integration delivered true end-to-end automation. These transformations expose a recurring architectural logic. Prompt Routing ensures every request reaches the most cost-efficient resource, grounding via (RAG) reduces hallucinations by anchoring outputs in verified knowledge, structured prompting standardizes behavior and tone across contexts, validation filters errors before they reach production, and integration converts correct responses into operational outcomes. Together, these layers convert isolated model calls into dependable systems. What follows is the complete blueprint, how these architectural principles interact as a unified production pipeline, transforming a single customer query into a reliable, verifiable transaction from end to end. 1.3 The Blueprint: How Production AI Systems Work Now that you've seen individual engineering techniques deliver results, let's examine how they work together as an integrated system. 1.3.1 Complete System Architecture
Page 17
Sarah tried buying concert tickets online. The payment failed with the error message “Transaction could not be completed”. Frustrated, she checked her bank account, the charge went through. Twice. Both charges are still showing as pending. She opens the company's support chat and types: “My payment failed but I was charged twice, can you help?” This single query travels through five distinct engineering layers before Sarah receives her response. Each layer serves a specific purpose in transforming her raw question into an accurate, helpful answer grounded in actual company policies. Figure 1.2 Production AI System Architecture
Page 18
What looks like complexity is careful simplicity, each layer does one thing well, and failures at any stage can be isolated and fixed without rebuilding the entire system. This architecture emerged from years of production AI development, where each component exists to prevent specific, costly failures that simpler approaches couldn't avoid.
Page 19
1.3.2 Following the Transaction Let's trace Sarah's query through each layer to see exactly how this prevents the disasters that cost the airline $3.2 million and how proper engineering delivers the reliability that Klarna achieves at scale. Sarah's query first hits the intelligent router, which analyzes the text to classify it as a PAYMENT_ISSUE with COMPLEX difficulty. This classification determines routing; the query goes to the full pipeline with RAG support and an advanced model. By contrast, a simple query like “What are your hours?” would route to a lightweight model at a fraction of the cost, roughly 1/50th the API expense. Without this routing layer, every query would hit the expensive models. The math is stark, proper prompt routing keeps monthly costs around $2,580, while sending all queries to premium models would create $15,000 monthly bills. The RAG pipeline activates next, searching the company's knowledge base for policies relevant to payment failures and duplicate charges. RAG (Retrieval Augmented Generation) RAG systems search external knowledge bases (like policy documents or manuals) and inject relevant information into prompts before generation. This grounds responses in authoritative sources rather than relying on the model's training data, dramatically reducing hallucinations. You'll learn to build RAG systems in Chapter 14. The system uses a Vector Database to find relevant documents by meaning (semantic similarity) rather than just keyword matching. Unlike traditional databases that search for exact phrases, Vector Databases identify that ‘payment failed but charged’ is semantically like documents about ‘duplicate charges’ or ‘authorization holds’, even when those exact words don't appear. This semantic search retrieves the top three matching documents, “Duplicate Charge Resolution Procedure”, “Payment Failure Handling Policy”, and “Refund Processing Guidelines”. These documents are formatted and injected into the prompt as authoritative context. Without RAG, the model would rely solely on its training data and general knowledge, leading to invented
Page 20
policies. A system without RAG might confidently state: “Refunds are processed within 90 days, credited in 24 hours, and you'll receive a $25 apology credit”. All incorrect. Sarah would receive promises that the company has no intention of keeping them because those policies don't exist. Response synthesis takes the retrieved context and generates an answer using carefully structured prompts. The system combines the authoritative policy documents with Sarah’s specific query, producing a response that acknowledges her frustration, explains what happened clearly, provides accurate timelines based on real policies, and cites the source documents. Without structured prompting, quality varies wildly, sometimes the tone is too cold and corporate, sometimes it overpromises to be helpful, sometimes it's vague and unhelpful. The validation layer runs four automated checks before the response reaches Sarah. Policy compliance verification ensures refund timelines match the official documents. Hallucination detection via Language Model as a Judge confirms that every factual claim appears in the source materials. Tone verification checks for appropriate empathy and professionalism. Citation validation ensures all referenced documents exist and were accurately quoted. This layer catches errors before customers see them. Without validation, problems slip through. In one real incident, an AI system promised “$50 gift card” compensation to 73 customers before anyone noticed the policy violation. The cost: $3,650 in unauthorized compensation that the company had to honor to maintain trust. The final step delivers Sarah's response with proper formatting, source citations, and metadata including a confidence score. Sarah's query receives a confidence score of 0.92 (High), which means immediate delivery. Had the score fallen below 0.70, the system would have triggered mandatory human review before sending any response. This confidence-based escalation ensures that uncertain or complex cases receive human attention while straightforward issues get resolved immediately. 1.3.3 The Five Engineering Layers
The above is a preview of the first 20 pages. Register to read the complete e-book.

Recommended for You

Loading recommended books...
Failed to load, please try again later

Tip the Site

Scan the WeChat Pay or Alipay code to tip. No login required.

WeChat Pay
Alipay
← Back to List