Test-Driven Development with Python Obey the Testing Goat, 3rd Edition (Harry Percival) (Z-Library)

Test-D riven D evelop m ent w ith Python Harry J.W. Percival Test-Driven Development with Python Obey the Testing Goat: Using Django, Selenium, and JavaScript 3rd Edition

9 7 8 1 0 9 8 1 4 8 7 1 3 5 7 9 9 9 ISBN: 978-1-098-14871-3 US $79.99 CAN $99.99 PROGR AMMING / PY THON The third edition of this trusted guide shows you how to apply test-driven development (TDD) to building real-world web applications with Python. By writing tests before building each part of your app—and then creating just enough code to pass them—you’ll learn how TDD leads to clean, reliable, and maintainable software. Author Harry J.W. Percival takes you through a practical, end-to-end example of web development using Python 3.14 and Django 5. Along the way, you’ll explore tools like Selenium, JavaScript, Git, and mocking, and discover how TDD supports better design decisions, encourages continuous improvement, and instills confidence in your codebase. Whether you’re a professional developer or just transitioning into web development, this book offers hands-on experience with modern testing workflows and architecture. • Follow the full TDD workflow, from writing tests first to refactoring with confidence • Write unit tests for core logic and functional tests for browser-based interactions • Use mock objects to isolate external systems and simplify integration • Package your application using Docker • Automate deployments and test your code in a staging environment • Validate third-party plug-ins and dependencies within your test suite • Set up continuous integration to run your tests automatically • Enrich your frontend with test-driven JavaScript Test-Driven Development with Python “Testing is essential for developer sanity. Harry does a fantastic job of holding our attention whilst exploring real-world testing practices.” Michael Foord, Python core developer and maintainer of unittest “This book is far more than an introduction to test-driven development— it’s a complete best-practices crash course, from start to f inish, on modern web application development with Python.” Kenneth Reitz, fellow at Python Software Foundation Harry J.W. Percival is a passionate advocate for TDD, sharing his expertise worldwide through talks and workshops. He works for Kraken Technologies, writing software to support the worldwide green energy transition. Test-D riven D evelop m ent w ith Python

Praise for Test-Driven Development with Python In this book, Harry takes us on an adventure of discovery with Python and testing. It’s an excellent book, fun to read, and full of vital information. It has my highest recommendations for anyone interested in testing with Python, learning Django, or wanting to use Selenium. Testing is essential for developer sanity and it’s a notoriously difficult field, full of trade-offs. Harry does a fantastic job of holding our attention whilst exploring real-world testing practices. —Michael Foord, Python Core Developer and Maintainer of unittest This book is far more than an introduction to test-driven development—it’s a complete best-practices crash course, from start to finish, into modern web application development with Python. Every web developer needs this book. —Kenneth Reitz, Fellow at Python Software Foundation Harry’s book is what we wish existed when we were learning Django. At a pace that’s achievable and yet delightfully challenging, it provides excellent instruction for Django and various test practices. The material on Selenium alone makes the book worth purchasing, but there’s so much more! —Daniel and Audrey Roy Greenfeld, authors of Two Scoops of Django (Two Scoops Press)

Harry J.W. Percival Test-Driven Development with Python Obey the Testing Goat: Using Django, Selenium, and JavaScript THIRD EDITION

978-1-098-14871-3 [LSI] Test-Driven Development with Python by Harry J.W. Percival Copyright © 2026 Harry Percival. All rights reserved. Published by O’Reilly Media, Inc., 141 Stony Circle, Suite 195, Santa Rosa, CA 95401. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (https://oreilly.com). For more information, contact our corporate/institu‐ tional sales department: 800-998-9938 or corporate@oreilly.com. Acquisitions Editor: Brian Guerin Development Editor: Rita Fernando Production Editor: Christopher Faucher Copyeditor: Piper Content Partners Proofreader: Kim Cofer Indexer: Ellen Troutman-Zaig Cover Designer: Susan Brown Cover Illustrator: Karen Montgomery Interior Designer: David Futato Interior Illustrator: Kate Dullea June 2014: First Edition August 2017: Second Edition October 2025: Third Edition Revision History for the Third Edition 2025-10-30: First Release See https://www.oreilly.com/catalog/errata.csp?isbn=0636920873884 for release details. The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Test-Driven Development with Python, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc. The views expressed in this work are those of the author, and do not represent the publisher’s views. While the publisher and the author have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights.

Table of Contents Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii Preface to the Third Edition: TDD in the Age of AI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxv Prerequisites and Assumptions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxix Acknowledgments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxxix Part I. The Basics of TDD and Django 1. Getting Django Set Up Using a Functional Test. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Obey the Testing Goat! Do Nothing Until You Have a Test 3 Getting Django Up and Running 7 Starting a Git Repository 10 2. Extending Our Functional Test Using the unittest Module. . . . . . . . . . . . . . . . . . . . . . . 15 Using a Functional Test to Scope Out a Minimum Viable App 16 The Python Standard Library’s unittest Module 19 Commit 22 3. Testing a Simple Home Page with Unit Tests. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 Our First Django App and Our First Unit Test 26 Unit Tests, and How They Differ from Functional Tests 26 Unit Testing in Django 28 Django’s MVC, URLs, and View Functions 29 v

Unit Testing a View 30 At Last! We Actually Write Some Application Code! 32 The Unit-Test/Code Cycle 33 Our Functional Tests Tell Us We’re Not Quite Done Yet 36 Reading Tracebacks 38 urls.py 40 4. What Are We Doing with All These Tests? (And, Refactoring). . . . . . . . . . . . . . . . . . . . . 45 Programming Is Like Pulling a Bucket of Water Up from a Well 46 Using Selenium to Test User Interactions 48 The “Don’t Test Constants” Rule, and Templates to the Rescue 51 Refactoring to Use a Template 52 Revisiting Our Unit Tests 55 Test Behaviour, Not Implementation 56 On Refactoring 58 A Little More of Our Front Page 60 Recap: The TDD Process 62 Double-Loop TDD 63 5. Saving User Input: Testing the Database. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 Wiring Up Our Form to Send a POST Request 66 Testing the Contract Between Frontend and Backend 66 Debugging Functional Tests 68 Debugging with time.sleep 69 Processing a POST Request on the Server 71 Passing Python Variables to Be Rendered in the Template 74 An Unexpected Failure 75 Improving Error Messages in Tests 76 Three Strikes and Refactor 79 The Django ORM and Our First Model 81 Our First Database Migration 83 The Test Gets Surprisingly Far 84 A New Field Means a New Migration 85 Saving the POST to the Database 86 Redirect After a POST 90 Better Unit Testing Practice: Each Test Should Test One Thing 93 Rendering Items in the Template 94 Creating Our Production Database with migrate 98 Recap 101 vi | Table of Contents

6. Improving Functional Tests: Ensuring Isolation and Removing Magic Sleeps. . . . . . 103 Ensuring Test Isolation in Functional Tests 104 Running Just the Unit Tests 107 On Implicit and Explicit Waits, and Magic time.sleeps 108 7. Working Incrementally. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 Small Design When Necessary 115 Not Big Design Up Front 115 YAGNI! 116 REST-ish 117 Implementing the New Design Incrementally Using TDD 118 Ensuring We Have a Regression Test 119 Iterating Towards the New Design 121 Taking a First, Self-Contained Step: One New URL 123 Separating Out Our Home Page and List View Functionality 123 The FTs Detect a Regression 126 Getting Back to a Working State as Quickly as Possible 128 Green? Refactor 130 Another Small Step: A Separate Template for Viewing Lists 131 A Third Small Step: A New URL for Adding List Items 135 A Test Class for New List Creation 136 A URL and View for New List Creation 137 Removing Now-Redundant Code and Tests 138 A Regression! Pointing Our Forms at the New URL 139 Debugging in DevTools 140 Biting the Bullet: Adjusting Our Models 143 A Foreign Key Relationship 145 Adjusting the Rest of the World to Our New Models 146 Each List Should Have Its Own URL 149 Capturing Parameters from URLs 150 Adjusting new_list to the New World 151 The Functional Tests Detect Another Regression 152 One More URL to Handle Adding Items to an Existing List 153 The Last New urls.py Entry 155 The Last New View 155 Testing Template Context Directly 157 A Final Refactor Using URL includes 160 Can You Believe It? 161 Table of Contents | vii

8. Prettification: Layout and Styling, and What to Test About It. . . . . . . . . . . . . . . . . . . 163 Testing Layout and Style 163 Prettification: Using a CSS Framework 167 Django Template Inheritance 169 Integrating Bootstrap 170 Rows and Columns 171 Static Files in Django 175 Switching to StaticLiveServerTestCase 176 Using Bootstrap Components to Improve the Look of the Site 177 Jumbotron! 177 Large Inputs 178 Table Styling 178 Optional: Dark Mode 179 A Semi-Decent Page 180 Parsing HTML for Less Brittle Tests of Key HTML Content 181 What We Glossed Over: collectstatic and Other Static Directories 185 A Few Things That Didn’t Make It 187 Part II. Going to Production 9. Containerization aka Docker. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 Docker, Containers, and Virtualization 193 Why Not Just Use a Virtualenv? 196 Docker and Your CV 196 As Always, Start with a Test 196 Making a src Folder 199 Installing Docker 199 Building a Docker Image and Running a Docker Container 202 A First Cut of a Dockerfile 202 Docker Build 204 Docker Run 205 Installing Django in a Virtualenv in Our Container Image 206 Successful Run 207 Using the FT to Check That Our Container Works 208 Debugging Container Networking Problems 209 Debugging Web Server Connectivity with curl 210 Running Code “Inside” the Container with docker exec 210 Docker Port Mapping 212 Essential Googling the Error Message 213 viii | Table of Contents

Database Migrations 217 Should We Run migrate Inside the Dockerfile? No. 219 Mounting Files Inside the Container 220 10. Making Our App Production-Ready. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223 What We Need to Do 223 Switching to Gunicorn 224 The FTs Catch a Problem with Static Files 225 Serving Static Files with WhiteNoise 226 Using requirements.txt 227 Using Environment Variables to Adjust Settings for Production 230 Setting DEBUG=True and SECRET_KEY 231 Setting Environment Variables Inside the Dockerfile 232 Setting Environment Variables at the Docker Command Line 232 ALLOWED_HOSTS Is Required When Debug Mode Is Turned Off 233 Collectstatic Is Required when Debug Is Turned Off 235 Switching to a Nonroot User 237 Making the Database Filepath Configurable 237 Using UIDs to Set Permissions Across Host/Container Mounts 238 Configuring Logging 240 Provoking a Deliberate Error 240 Exercise for the Reader: Using the Django check Command 242 Wrap-Up 243 11. Getting a Server Ready for Deployment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245 Manually Provisioning a Server to Host Our Site 245 Choosing Where to Host Our Site 246 Spinning Up Our Own Server 246 Getting a Domain Name 247 Configuring DNS for Staging and Live Domains 248 Ansible 249 Ansible Versus SSH: How We’ll Talk to Our Server 249 Start by Making Sure We Can SSH In 250 Debugging Issues with SSH 251 Installing Ansible 253 Checking Ansible Can Talk to Our Server 254 12. Infrastructure as Code: Automated Deployments with Ansible. . . . . . . . . . . . . . . . . . 257 A First Cut of an Ansible Playbook for Deployment 258 SSHing Into the Server and Viewing Container Logs 261 Allowing Rootless Docker Access 263 Table of Contents | ix

Getting Our Image Onto the Server 265 Taking a Look Around Manually 269 Setting Environment Variables and Secrets 270 Manually Checking Environment Variables for Running Containers 273 Running FTs to Check on Our Deploy 275 Manual Debugging with curl Against the Staging Server 275 Mounting the Database on the Server and Running Migrations 277 It Workssss 279 Deploying to Prod 279 git tag the Release 280 Tell Everyone! 280 Further Reading 281 Part III. Forms and Validation 13. Splitting Our Tests into Multiple Files, and a Generic Wait Helper. . . . . . . . . . . . . . . . 285 Start on a Validation FT: Preventing Blank Items 285 Skipping a Test 286 Splitting Functional Tests Out into Many Files 288 Running a Single Test File 291 A New FT Tool: A Generic Explicit Wait Helper 291 Finishing Off the FT 296 Refactoring Unit Tests into Several Files 297 14. Validation at the Database Layer. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301 Model-Layer Validation 302 The self.assertRaises Context Manager 303 Django Model Constraints and Their Interaction with Databases 303 Inspecting Our Constraints at the Database Level 304 Testing Django Model Validation 305 A Django Quirk: Model Save Doesn’t Run Validation 305 Surfacing Model Validation Errors in the View 307 Checking That Invalid Input Isn’t Saved to the Database 310 Adding an Early Return to Our FT to Let Us Refactor Against Green 312 Django Pattern: Processing POST Requests in the Same View That Renders the Form 313 Refactor: Transferring the new_item Functionality into view_list 313 Enforcing Model Validation in view_list 318 Refactor: Removing Hardcoded URLs 320 x | Table of Contents

The {% url %} Template Tag 320 Using get_absolute_url for Redirects 321 15. A Simple Form. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325 Moving Validation Logic Into a Form 325 Exploring the Forms API with a Unit Test 326 Switching to a Django ModelForm 328 Testing and Customising Form Validation 330 Attempting to Use the Form in Our Views 332 Using the Form in a View with a GET Request 332 The Trade-offs of Django ModelForms: The Frontend Is Coupled to the Database 334 A Big Find-and-Replace 335 Backing Out Our Changes and Getting to a Working State 336 Renaming the name Attribute 337 Renaming the id Attribute 340 A Second Attempt at Using the Form in Our Views 342 Using the Form in a View That Takes POST Requests 345 Using the Form to Display Errors in the Template 346 Get Back to a Working State 347 A Helper Method for Several Short Tests 348 Using the Form in the Existing Lists View 351 Using the Form to Pass Errors to the Template 351 Refactoring the View to Use the Form Fully 353 An Unexpected Benefit: Free Client-Side Validation from HTML5 356 A Pat on the Back 358 But Have We Wasted a Lot of Time? 358 Using the ModelForm’s Own Save Method 359 16. More Advanced Forms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363 Another FT for Duplicate Items 364 Preventing Duplicates at the Model Layer 365 Rewriting the Old Model Test 367 Integrity Errors That Show Up on Save 368 Experimenting with Duplicate Item Validation at the Views Layer 371 A More Complex Form to Handle Uniqueness Validation 373 Using the Existing List Item Form in the List View 375 Customising the Save Method on Our New Form 376 The FTs Pick Up an Issue with Bootstrap Classes 378 Conditionally Customising CSS Classes for Invalid Forms 379 A Little Digression on Queryset Ordering and String Representations 381 Table of Contents | xi

On the Trade-offs of Django ModelForms, and Frameworks in General 384 Moving Presentation Logic Back into the Template 386 Tidying Up the Forms 392 Switching Back to Simple Forms 393 Wrapping Up: What We’ve Learned About Testing Django 395 Part IV. More Advanced Topics in Testing 17. A Gentle Excursion into JavaScript. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399 Starting with an FT 400 A Quick Spike 402 A Simple Inline Script 403 Using the Browser DevTools 405 Choosing a Basic JavaScript Test Runner 407 An Overview of Jasmine 408 Setting Up Our JavaScript Test Environment 409 Our First Smoke Test: Describe, It, Expect 410 Running the Tests via the Browser 410 Testing with Some DOM Content 412 Building a JavaScript Unit Test for Our Desired Functionality 415 Fixtures, Execution Order, and Global State: Key Challenges of JavaScript Testing 417 console.log for Debug Printing 418 Using an Initialize Function for More Control Over Execution Time 420 Deliberately Breaking Our Code to Force Ourselves to Write More Tests 421 Red/Green/Refactor: Removing Hardcoded Selectors 422 Does it Work? 426 Testing Integration with CSS and Bootstrap 427 Columbo Says: Wait for Onload 430 JavaScript Testing in the TDD Cycle 431 18. Deploying Our New Code. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433 The Deployment Checklist 433 A Full Test Run Locally 434 Quick Test Run Against Docker 434 Staging Deploy and Test Run 436 Production Deploy 437 What to Do If You See a Database Error 437 How to Delete the Database on the Staging Server 437 xii | Table of Contents

Wrap-Up: git tag the New Release 438 19. User Authentication, Spiking, and De-Spiking. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439 Passwordless Auth with “Magic Links” 439 A Somewhat Larger Spike 440 Starting a Branch for the Spike 441 Frontend Login UI 441 Sending Emails from Django 442 Email Server Config for Django 444 Another Secret, Another Environment Variable 444 Storing Tokens in the Database 445 Custom Authentication Models 447 Finishing the Custom Django Auth 448 De-Spiking 453 Making a Plan 453 Wring an FT Against the Spiked Code 453 Reverting Our Spiked Code 455 A Minimal Custom User Model 459 Tests as Documentation 463 A Token Model to Link Emails with a Unique ID 465 20. Using Mocks to Test External Dependencies. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 469 Before We Start: Getting the Basic Plumbing In 470 Mocking Manually—aka Monkeypatching 471 The Python Mock Library 475 Using unittest.patch 476 Getting the FT a Little Further Along 478 Testing the Django Messages Framework 479 Adding Messages to Our HTML 482 Starting on the Login URL 483 Checking That We Send the User a Link with a Token 484 De-Spiking Our Custom Authentication Backend 487 One if = One More Test 488 The get_user Method 491 21. Using Mocks for Test Isolation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 495 Using Our Auth Backend in the Login View 496 Straightforward Non-Mocky Test for Our View 498 Combinatorial Explosion 500 The Car Factory Example 500 Using Mocks to Test Parts of Our System in Isolation 503 Table of Contents | xiii

Mocks Can Also Let You Test the Implementation, When It Matters 504 Starting Again: Test-Driving Our Implementation with Mocks 505 Using mock.return_value 509 Using .return_value During Test Setup 512 UnDONTifying 514 Deciding Which Tests to Keep 515 The Moment of Truth: Will the FT Pass? 517 It Works in Theory! Does It Work in Practice? 518 Using Our New Environment Variable, and Saving It to .env 518 Finishing Off Our FT: Testing Logout 521 22. Test Fixtures and a Decorator for Explicit Waits. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 525 Skipping the Login Process by Pre-creating a Session 526 Checking That It Works 528 Our Final Explicit Wait Helper: A Wait Decorator 530 23. Debugging and Testing Server Issues. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 537 The Proof Is in the Pudding: Using Docker to Catch Final Bugs 537 Inspecting the Docker Container Logs 538 Another Environment Variable in Docker 540 mail.outbox Won’t Work Outside Django’s Test Environment 541 Deciding How to Test “Real” Email Sending 541 An Alternative Method for Setting Secret Environment Variables on the Server 544 Debugging with SQL 545 Managing Fixtures in Real Databases 546 A Django Management Command to Create Sessions 547 Getting the FT to Run the Management Command on the Server 549 Running Commands Using Docker Exec and (Optionally) SSH 549 Recap: Creating Sessions Locally Versus Staging 551 Testing the Management Command 552 Test Database Cleanup 554 Wrap-Up 555 24. Finishing “My Lists”: Outside-In TDD. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 557 The Alternative: Inside-Out 557 Why Prefer “Outside-In”? 558 The FT for “My Lists” 558 The Outside Layer: Presentation and Templates 562 Moving Down One Layer to View Functions (the Controller) 563 Another Pass, Outside-In 565 xiv | Table of Contents

A Quick Restructure of Our Template Composition 565 An Early Return So We’re Refactoring Against Green 567 Factoring Out Two Template includes 568 Designing Our API Using the Template 572 Moving Down to the Next Layer: What the View Passes to the Template 575 The Next “Requirement” from the Views Layer: New Lists Should Record Owner 576 A Decision Point: Whether to Proceed to the Next Layer with a Failing Test 577 Moving Down to the Model Layer 577 Final Step: Feeding Through the .name API from the Template 580 25. CI: Continuous Integration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 583 CI in Modern Development Workflows 584 Choosing a CI Service 584 Getting Our Code into GitLab 585 Signing Up 585 Starting a Project 585 Pushing Our Code Up Using Git Push 586 Setting Up a First Cut of a CI Pipeline 587 First Build! (and First Failure) 588 Trying to Reproduce a CI Error Locally 592 Enabling Debug Logs for Selenium/Firefox/Webdriver 594 Enabling Headless Mode for Firefox 596 A Common Bugbear: Flaky Tests 597 Taking Screenshots 597 Saving Build Outputs (or Debug Files) as Artifacts 599 If in Doubt, Try Bumping the Timeout! 601 A Successful Python Test Run 602 Running Our JavaScript Tests in CI 602 Installing Node.js 603 Installing and Configuring the Jasmine Browser Runner 603 Adding a Build Step for JavaScript 607 Tests Now Pass 609 Some Things We Didn’t Cover 610 Defining a Docker Image for CI 610 Caching 610 Automated Deployment, aka Continuous Delivery (CD) 610 26. The Token Social Bit, the Page Pattern, and an Exercise for the Reader. . . . . . . . . . . 613 An FT with Multiple Users, and addCleanup 614 The Page Pattern 615 Table of Contents | xv

Extend the FT to a Second User, and the “My Lists” Page 618 An Exercise for the Reader 620 Step-by-Step Guide 620 27. Fast Tests, Slow Tests, and Hot Lava. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 625 Why Do We Test? Our Desiderata for Effective Tests 626 Confidence and Correctness (Preventing Regression) 627 A Productive Workflow 627 Driving Better Design 627 Were Our Unit Tests Integration Tests All Along? What Is That Warm Glow Coming from the Database? 627 We’ve Been in the “Sweet Spot” 628 What Is a “True” Unit Test? Does it Matter? 628 Integration and Functional Tests Get Slower Over Time 628 We’re Not Getting the Full Potential Benefits of Testing 629 The Ideal of the Test Pyramid 630 Avoiding Mock Hell 631 The Actual Solutions Are Architectural 632 Ports and Adapters/Hexagonal/Onion/Clean Architecture 633 Functional Core, Imperative Shell 634 The Central Conceit: These Architectures Are “Better” 634 The Hardest Part: Knowing When to Make the Switch 635 Wrap-Up 636 Further Reading 637 Obey the Testing Goat!. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 639 Bibliography. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 641 A. Cheat Sheet. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 643 B. What to Do Next. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 647 C. Source Code Examples. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 651 Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 655 xvi | Table of Contents

Preface This book has been my attempt to share with the world the journey I took from “hacking” to “software engineering”. It’s mainly about testing, but there’s a lot more to it, as you’ll soon see. I want to thank you for reading it. If you bought a copy, then I’m very grateful. If you’re reading the free online version, then I’m still grateful that you’ve decided it’s worth spending your time on. Who knows; perhaps once you get to the end, you’ll decide it’s good enough to buy a physical copy for yourself or a friend. If you have any comments, questions, or suggestions, I’d love to hear from you. You can reach me directly via obeythetestinggoat@gmail.com, or on Mastodon @hjwp. You can also check out the website and my blog. I hope you’ll enjoy reading this book as much as I enjoyed writing it. Why I Wrote a Book About Test-Driven Development “Who are you, why have you written this book, and why should I read it?” I hear you ask. I was lucky enough early on in my career to fall in with a bunch of test-driven development (TDD) fanatics, and it made such a big impact on my programming that I was burning to share it with everyone. You might say I had the enthusiasm of a recent convert, and the learning experience was still a recent memory for me, so that’s what led to the first edition, back in 2014. When I first learned Python (from Mark Pilgrim’s excellent Dive Into Python), I came across the concept of TDD, and thought, “Yes. I can definitely see the sense in that”. Perhaps you had a similar reaction when you first heard about TDD? It seemed like a really sensible approach, a really good habit to get into—like regularly flossing your teeth. xvii

Then came my first big project, and you can guess what happened—there was a client, there were deadlines, there was lots to do, and any good intentions about TDD went straight out of the window. And, actually, it was fine. I was fine. At first. At first I thought I didn’t really need TDD because the website was small, and I could easily test whether things worked by just manually checking it out. Click this link here, choose that drop-down item there, and this should happen. Easy. This whole “writing tests” thing sounded like it would have taken ages. And besides, I fancied myself, from the full height of my three weeks of adult coding experience, as being a pretty good programmer. I could handle it. Easy. Then came the fearful goddess Complexity. She soon showed me the limits of my experience. The project grew. Parts of the system started to depend on other parts. I did my best to follow good principles like DRY (don’t repeat yourself), but that just led to some pretty dangerous territory. Soon, I was playing with multiple inheritance. Class hierarchies eight levels deep. eval statements. I became scared of making changes to my code. I was no longer sure what depended on what, and what might happen if I changed this code over here…oh gosh, I think that bit over there inherits from it…no, it doesn’t; it’s overridden. Oh, but it depends on that class variable. Right, well, as long as I override the override it should be fine. I’ll just check—but checking was getting much harder. There were lots of sections for the site now, and clicking through them all manually was starting to get impractical. Better to leave well enough alone. Forget refactoring. Just make do. Soon I had a hideous, ugly mess of code. New development became painful. Not too long after this, I was lucky enough to get a job with a company called Resolver Systems (now PythonAnywhere), where extreme programming (XP) was the norm. The people there introduced me to rigorous TDD. Although my previous experience had certainly opened my mind to the possible benefits of automated testing, I still dragged my feet at every stage. “I mean, testing in general might be a good idea, but really? All these tests? Some of them seem like a total waste of time…what? Functional tests as well as unit tests? Come on, that’s overdoing it! And this TDD test/minimal-code-change/test cycle? This is just silly! We don’t need all these baby steps! Come on—we can see what the right answer is; why don’t we just skip to the end?” Believe me, I second-guessed every rule, I suggested every shortcut, I demanded jus‐ tifications for every seemingly pointless aspect of TDD—and I still came out seeing xviii | Preface