Practical Python Data Wrangling and Data Quality Getting Started with Reading, Cleaning, and Analyzing Data (Susan E. McGregor)（Z-Library）

# Practical Python Data Wrangling and Data Quality ## 【One-Line Pitch】 A hands-on guide for journalists, analysts, and researchers who need to turn messy, real-world data into clean, trustworthy datasets using Python—covering everything from data quality assessment to code refactoring. If you've ever felt that your data is "dirty" and you're not sure where to start, this book is your practical roadmap. ## 【Book Arc】 - **Opening (~0%–10%)**: Introduces the core philosophy that data is inherently human-made and therefore flawed, establishing the two key axes of data quality: integrity (how well the data is structured and documented) and fit (how appropriate it is for your specific question). Sets up the book's practical, project-based approach. - **Early (~10%–25%)**: Walks through setting up a complete Python environment (Miniconda, Jupyter Notebook, Atom editor) and establishing a "data wrangling diary" workflow—emphasizing the importance of writing your research question as a single sentence and using version control (Git/GitHub) to track all changes. - **Early (~25%–35%)**: Covers Python fundamentals through the lens of data wrangling, introducing the five core data types (numbers, strings, lists, dictionaries, booleans) and the crucial concept that "nouns ≈ variables"—using syntax highlighting and punctuation to reliably identify data types. - **Middle (~35%–50%)**: Dives into programming structures—custom functions (described as "recipes"), loops, conditionals, and nesting—with a strong emphasis on Python's whitespace-dependent indentation. Introduces exception handling but argues that for data quality work, you often *want* errors to surface rather than silently pass over bad data. - **Middle (~50%–70%)**: Applies these skills to real data wrangling tasks, using the Citi Bike dataset as a running example. Covers selecting data subsets, regular expressions for string matching, handling dates, cleaning fixed-width files, correcting spelling inconsistencies, and augmenting data with additional sources. - **Late (~70%–100%)**: Focuses on structuring and refactoring code for maintainability, revisiting custom functions with criteria like "will you use it more than once?" and "is it ugly and confusing?"—plus a comprehensive framework for assessing data quality across dimensions like annotation, volume, consistency, atomicity, clarity, and dimensionality. ## 【Key Takeaways】 - **Data quality has two distinct axes: integrity and fit** (Early): Integrity concerns the data's internal characteristics (completeness, atomicity, annotation), while fit asks whether the data actually answers your question. Both matter, and neither is guaranteed by sophisticated tools. - **Computers amplify human judgment—they don't replace it** (Early): Even "intelligent" systems are just pattern-matching on human-selected data. The responsibility for data quality rests with the humans who collect, clean, and analyze it. - **Write your research question as a single sentence** (Early): This simple discipline prevents you from losing track of your goal when data wrangling inevitably leads you down "rabbit holes." Your question becomes the anchor for all subsequent decisions. - **Python's five data types are identifiable by punctuation alone** (Early): Numbers are just digits, strings have matching quotes, lists use square brackets, dictionaries use braces, and booleans are True/False. This makes type identification reliable and beginner-friendly. - **Indentation is not optional in Python—it's structural** (Middle): Unlike curly-brace languages, Python uses whitespace to define code blocks. Understanding nesting (one tab per level) is essential for writing loops, conditionals, and functions that work correctly. - **Exceptions are your friends in data quality work** (Middle): While you *can* write code to handle errors gracefully, the book argues you often *shouldn't*—because silent error handling can hide data quality problems you need to know about. - **Comments are for your future self, not the computer** (Middle): The book's examples deliberately include more comment lines than code, because you *will* forget what your code does and why. Detailed comments are a sign of good practice, not verbosity. - **Data cleaning is rarely a straight line** (Middle): The "circuitous path to simple solutions" is normal—real data has spelling inconsistencies, weird date formats, and fixed-width files that need conversion. Expect detours and build them into your workflow. ## 【Reading Tips】 - **Skim the setup chapters (10%–25%) if you already have Python installed**: The Miniconda/Jupyter/Atom installation walkthrough is thorough but platform-specific; focus instead on the "data diary" concept and Git workflow, which are transferable habits. - **Deep-read the Citi Bike examples (Middle section)**: The running example of counting Subscriber vs. Customer user types is deceptively simple—it demonstrates the full loop of reading data, transforming it, and checking for unexpected values. This is where the book's philosophy becomes concrete. - **Pay special attention to the "Gotchas" sections**: These are the hard-won lessons from real data wrangling—things like empty values not converting to zero, or Excel dates needing decryption. These will save you hours of debugging later. - **Don't skip the code comments**: The book deliberately includes extensive inline comments in its examples. Reading them is like having the author explain her thinking process, which is more valuable than just copying the code. - **Use the data quality framework as a checklist**: The late chapters on assessing data fit (validity, reliability, representativeness) and structural quality (atomicity, clarity, dimensionality) are worth returning to as a reference whenever you start a new dataset. ## 【Coverage Limits】 This guide covers the book's core philosophy, setup, Python fundamentals, and data cleaning approach through the middle sections. The excerpts do not cover the later chapters on advanced data augmentation techniques, code refactoring in depth, or the final data quality assessment frameworks in detail—those sections are summarized from the table of contents rather than excerpted content. ##