Red Teaming AI Attacking Defending Intelligent Systems (AI Security Book 1) (Philip A. Dursey) (Z-Library)

# Red Teaming AI: Attacking & Defending Intelligent Systems ## 【One-Line Pitch】 A comprehensive field manual for security professionals and AI practitioners who need to think like attackers to defend intelligent systems—covering everything from data poisoning and model extraction to prompt injection and deepfake threats. If you build, deploy, or secure AI systems, this 1100+ page guide gives you the adversarial mindset and practical techniques to find vulnerabilities before real attackers do. ## 【Book Arc】 - **Opening (~0%–10%)**: Establishes why AI systems demand a fundamentally different security approach than traditional software, introducing the core concepts of AI Red Teaming as a structured discipline. Covers the unique threat landscape, the adversarial mindset, and the methodology that underpins the entire book. - **Early (~10%–23%)**: Dives into the technical attack surface—threat modeling frameworks, attacker capability assessment (white-box vs. black-box vs. gray-box knowledge), and the first major attack categories including data poisoning, label flipping, and clean-label attacks. Introduces the ML data pipeline as an expanded attack surface and covers online/federated learning risks. - **Early (~23%–32%)**: Moves into model extraction and inversion techniques, including active learning approaches for building substitute models, membership inference attacks, and defenses like differential privacy and knowledge distillation. Transitions into LLM-specific attacks with prompt injection, jailbreaking, and obfuscation techniques. - **Middle (~39%–48%)**: Expands to infrastructure and supply chain security for AI systems—software dependencies, compromised pre-trained models, malicious datasets, and CI/CD pipeline attacks. Covers model inversion attacks in depth, including exploiting confidence scores and generative model vulnerabilities, plus federated learning attack vectors. - **Late (~48%–end)**: Addresses the broader strategic landscape: how adversaries weaponize generative AI for phishing and social engineering at scale, deepfake threats, and defense-in-depth mitigation strategies. Connects technical red teaming to organizational risk management, regulation, and ethics. ## 【Key Takeaways】 - **AI security requires a different playbook than traditional security** (Early): Conventional defenses fail against attacks targeting the intelligence and learning capabilities of AI systems themselves. The book establishes this foundational premise before introducing AI-specific red teaming as a structured discipline. - **Adversary knowledge level dictates attack feasibility** (Early): Whether an attacker operates with white-box (full model access), gray-box (partial), or black-box (inputs/outputs only) knowledge fundamentally shapes which attacks are possible. Red teams must assess this early in threat modeling to scope their testing properly. - **Data poisoning attacks exploit the training pipeline** (Early): Label flipping, clean-label attacks, and poisoned samples injected before training can shift decision boundaries without being detected. Systems that learn continuously (online learning) or from distributed sources (federated learning) face heightened poisoning risks. - **Model extraction is a practical, budget-conscious attack** (Early): Attackers can build substitute models using active learning strategies—uncertainty sampling, query-by-committee—to replicate target model behavior within API query budgets. Defenses include tiered access, adaptive rate limiting, and IP/account fingerprinting. - **Prompt injection and jailbreaking are distinct LLM threats** (Early): Obfuscation testing (Base64, leetspeak, Unicode homoglyphs) and few-shot learning exploitation can bypass input filters and safety rules. The book distinguishes prompt injection from jailbreaking and provides systematic testing techniques for both. - **AI supply chains are a critical attack surface** (Middle): Compromised dependencies, malicious pre-trained models, unsafe deserialization, and poisoned datasets can inject backdoors that bypass perimeter defenses. Red teams should analyze SBOMs, check for dependency confusion, and audit build pipeline security. - **Model inversion can leak training data features** (Middle): Even classifiers, not just generative models, can leak representative data through confidence score exploitation and hill-climbing attacks. Real-world case studies show hospitals and other organizations facing severe privacy breaches from these techniques. - **Defense requires layered, adaptive strategies** (Late): No single defense is foolproof—certified defenses offer formal guarantees but are computationally expensive, while detection methods can be evaded by adaptive attackers. Defense-in-depth combining regularization, differential privacy, output controls, and monitoring is the recommended approach. ## 【Reading Tips】 - **Deep-read Part I (Foundations)** for the adversarial mindset and methodology—this conceptual framework underpins every technique in the book. Skim the reference lists at chapter ends unless you need academic sources for your own work. - **Focus on the practical attack techniques in the Early sections** (data poisoning, model extraction, prompt injection)—these are immediately actionable and include code listings and step-by-step testing procedures. The active learning code example for model extraction is worth studying carefully. - **Pay special attention to the defense sections paired with each attack**—the book consistently presents pros and cons for each defense (e.g., certified defenses vs. detection methods), which helps you make informed trade-off decisions for your specific use case. - **The Middle sections on supply chain security** are essential reading if you're responsible for AI infrastructure, not just model development. The discussion of SBOMs, dependency confusion, and compromised pre-trained models is particularly relevant for production systems. - **Skim the LLM-specific attack sections if you work primarily with traditional ML models**, but note that the book argues these techniques increasingly apply across model types. The dual-use concerns and ethical considerations in the Late sections are worth reading regardless of your technical focus. ## 【Coverage Limits】 This guide synthesizes the first half of the book (approximately 0–48%), covering foundations, core attack techniques, and infrastructure security. The Late sections on defense integration, regulation, ethics, and societal impact are only partially covered in the available excerpts. ##