大模型基础 (毛玉仁，高云君) (Z Library)

【One-Line Pitch】 A systematic, open-source textbook on large language models (LLMs) that walks readers from classical language modeling through modern architectures, prompt engineering, efficient fine-tuning, model editing, and retrieval-augmented generation—ideal for students, researchers, and practitioners who want a rigorous yet accessible foundation in LLM technology. 【Book Arc】 - **Opening (~0%–15%)**: Introduces the book's mission, author contributions, and chapter roadmap, then lays the groundwork with classical language models—covering statistical n-grams, RNN-based models, and Transformer-based models—along with sampling methods and evaluation metrics (intrinsic and extrinsic). - **Early (~15%–40%)**: Explores LLM architectures in depth, starting with the "big data + big model → new intelligence" paradigm, then comparing encoder-only (BERT), encoder-decoder (T5, BART), and decoder-only (GPT, LLaMA) designs, plus non-Transformer alternatives like state space models (SSM) and test-time training (TTT). - **Middle (~40%–67%)**: Dives into prompt engineering—defining prompts, in-context learning, chain-of-thought (CoT) reasoning, and practical prompting techniques—then applies these to real-world use cases such as LLM agents, data synthesis, Text-to-SQL, and GPTs. - **Late (~67%–90%)**: Covers parameter-efficient fine-tuning (PEFT), explaining why it matters for downstream task adaptation, and detailing methods like parameter addition (input/model/output), parameter selection, and low-rank adaptation (LoRA) with its variants and task generalization. - **Ending (~90%–100%)**: Focuses on model editing—its philosophy, definitions, properties, and datasets—then compares classic approaches (external extension vs. internal modification) and walks through specific techniques like T-Patcher (additional parameters) and ROME (locate-and-edit). 【Key Takeaways】 - **Language models evolved from statistics to deep learning** (Opening): The book traces n-gram models, RNNs, and Transformers, showing how each step addressed prior limitations—essential for understanding why LLMs work the way they do. - **Architecture choice determines model capabilities** (Early): Encoder-only, encoder-decoder, and decoder-only designs serve different tasks; the book compares them functionally and historically, helping readers pick the right backbone for their use case. - **Non-Transformer architectures are emerging alternatives** (Early): SSMs and TTT models challenge the dominance of Transformers, offering new efficiency and adaptability trade-offs worth tracking for future-proofing skills. - **Prompt engineering is a skill with structure, not magic** (Middle): From in-context learning to chain-of-thought, the book breaks down how to select demonstrations, influence performance, and apply techniques like step-by-step reasoning and psychological cues. - **Prompts unlock practical applications** (Middle): Real-world uses like LLM agents, data synthesis, and Text-to-SQL show how prompt design translates into tangible products, bridging theory and deployment. - **Parameter-efficient fine-tuning saves resources without sacrificing quality** (Late): PEFT methods—especially LoRA—let you adapt large models to downstream tasks with minimal trainable parameters, making customization feasible on limited hardware. - **Model editing offers surgical updates without full retraining** (Ending): Techniques like T-Patcher and ROME modify specific knowledge or behaviors, balancing precision, generalization, and robustness—critical for maintaining models post-deployment. 【Reading Tips】 - **Skim the opening chapter** if you're already familiar with n-grams, RNNs, and Transformers; focus instead on the sampling and evaluation sections, which are practical and often glossed over elsewhere. - **Deep-read Chapter 2** (architecture) if you're choosing a model for a project—the functional comparison and historical evolution will help you justify your choice. - **Treat Chapter 3 (prompt engineering) as a hands-on playbook**: Try the techniques (e.g., CoT, demonstration selection) on your own tasks, and revisit the application section for agent and Text-to-SQL patterns. - **For PEFT and model editing (Chapters 4–5), prioritize the method taxonomies** (e.g., parameter addition vs. selection; external vs. internal editing) over individual papers; the book's structure helps you categorize new research you'll encounter later. - **Use the Paper List per chapter** as a reading list to go deeper—the book is designed as a living document, so pair it with recent literature for cutting-edge updates. 【Coverage Limits】 This guide synthesizes the table of contents and chapter structure; detailed technical content (e.g., specific formulas, code examples, or full method walkthroughs) is not covered here, and the excerpts do not include the book's final chapters on retrieval-augmented generation or future topics like inference acceleration and agents.