Registered users can read the full content for free
Register as a Gaohf Library member to read the complete e-book online for free and enjoy a better reading experience.
Page
1
t.me/DevTwitter
Page
2
Clean Code in Python Second Edition Develop maintainable and efficient code Mariano Anaya BIRMINGHAM - MUMBAI t.me/DevTwitter
Page
3
Clean Code in Python Second Edition Copyright © 2020 Packt Publishing All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information. Producers: Tushar Gupta Acquisition Editor – Peer Reviews: Divya Mudaliar Content Development Editor: Bhavesh Amin Technical Editor: Karan Sonawane Project Editor: Mrunal Dave Copy Editor: Safis Editing Proofreader: Safis Editing Indexer: Tejal Soni Presentation Designer: Ganesh Bhadwalkar First published: August 2018 Second Edition: December 2020 Production reference: 2281220 Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK. ISBN 978-1-80056-021-5 www.packt.com t.me/DevTwitter
Page
4
packt.com Subscribe to our online digital library for full access to over 7,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website. Why subscribe? • Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionals • Learn better with Skill Plans built especially for you • Get a free eBook or video every month • Fully searchable for easy access to vital information • Copy and paste, print, and bookmark content Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.Packt.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at customercare@packtpub.com for more details. At www.Packt.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks. t.me/DevTwitter
Page
5
Contributors About the author Mariano Anaya is a software engineer who spends most of his time creating software and mentoring fellow programmers. Mariano's primary areas of interest are software architecture, functional programming, and distributed systems. He was a speaker at Euro Python in 2016 and 2017, and FOSDEM 2019. To find out more about him, you can refer to his GitHub account with the username rmariano. Dedicated to my family and friends, for their unconditional support. t.me/DevTwitter
Page
6
About the reviewer Tarek Ziadé has been a Python developer for a long time. Back in the old days, he founded the French Python User group (AFPY) and was involved in the language development around packaging. He has written books about Python in both French (his native language) and English. Tarek worked at Mozilla for over a decade, building tools and services, leveraging Python's awesomeness in projects at scale. He now works as a principal engineer at Elastic. t.me/DevTwitter
Page
7
t.me/DevTwitter
Page
8
[ i ] Table of Contents Preface ix Chapter 1: Introduction, Code Formatting, and Tools 1 Introduction 2 The meaning of clean code 2 The importance of having clean code 3 Some exceptions 5 Code formatting 6 Adhering to a coding style guide on your project 6 Documentation 9 Code comments 9 Docstrings 10 Annotations 13 Do annotations replace docstrings? 17 Tooling 19 Checking type consistency 20 Generic validations in code 22 Automatic formatting 23 Setup for automatic checks 26 Summary 28 References 28 Chapter 2: Pythonic Code 29 Indexes and slices 30 Creating your own sequences 32 Context managers 34 Implementing context managers 37 Comprehensions and assignment expressions 40 t.me/DevTwitter
Page
9
Table of Contents [ ii ] Properties, attributes, and different types of methods for objects 43 Underscores in Python 44 Properties 46 Creating classes with a more compact syntax 49 Iterable objects 52 Creating iterable objects 53 Creating sequences 56 Container objects 57 Dynamic attributes for objects 59 Callable objects 61 Summary of magic methods 63 Caveats in Python 64 Mutable default arguments 64 Extending built-in types 66 A brief introduction to asynchronous code 68 Summary 70 References 71 Chapter 3: General Traits of Good Code 73 Design by contract 74 Preconditions 76 Postconditions 76 Pythonic contracts 77 Design by contract – conclusions 77 Defensive programming 78 Error handling 78 Value substitution 79 Exception handling 80 Using assertions in Python 88 Separation of concerns 90 Cohesion and coupling 91 Acronyms to live by 92 DRY/OAOO 92 YAGNI 94 KIS 95 EAFP/LBYL 97 Inheritance in Python 99 When inheritance is a good decision 99 Anti-patterns for inheritance 100 Multiple inheritance in Python 103 Method Resolution Order (MRO) 104 Mixins 106 Arguments in functions and methods 107 t.me/DevTwitter
Page
10
Table of Contents [ iii ] How function arguments work in Python 107 How arguments are copied to functions 107 Variable number of arguments 109 Positional-only parameters 114 Keyword-only arguments 115 The number of arguments in functions 117 Function arguments and coupling 117 Compact function signatures that take too many arguments 118 Final remarks on good practices for software design 120 Orthogonality in software 120 Structuring the code 122 Summary 123 References 124 Chapter 4: The SOLID Principles 125 The single responsibility principle 126 A class with too many responsibilities 126 Distributing responsibilities 128 The open/closed principle 129 Example of maintainability perils for not following the OCP 130 Refactoring the events system for extensibility 133 Extending the events system 135 Final thoughts about the OCP 137 Liskov's substitution principle 137 Detecting LSP issues with tools 138 Using mypy to detect incorrect method signatures 138 Detecting incompatible signatures with pylint 140 More subtle cases of LSP violations 141 Remarks on the LSP 144 Interface segregation 144 An interface that provides too much 146 The smaller the interface, the better 146 How small should an interface be? 148 Dependency inversion 149 A case of rigid dependencies 150 Inverting the dependencies 150 Dependency injection 152 Summary 154 References 155 Chapter 5: Using Decorators to Improve Our Code 157 What are decorators in Python? 158 Function decorators 159 Decorators for classes 160 t.me/DevTwitter
Page
11
Table of Contents [ iv ] Other types of decorator 165 More advanced decorators 165 Passing arguments to decorators 165 Decorators with nested functions 166 Decorator objects 168 Decorators with default values 169 Decorators for coroutines 172 Extended syntax for decorators 175 Good uses for decorators 176 Adapting function signatures 176 Validating parameters 178 Tracing code 178 Effective decorators – avoiding common mistakes 178 Preserving data about the original wrapped object 179 Dealing with side effects in decorators 181 Incorrect handling of side effects in a decorator 182 Requiring decorators with side effects 184 Creating decorators that will always work 186 Decorators and clean code 189 Composition over inheritance 189 The DRY principle with decorators 192 Decorators and separation of concerns 193 Analysis of good decorators 195 Summary 197 References 197 Chapter 6: Getting More Out of Our Objects with Descriptors 199 A first look at descriptors 200 The machinery behind descriptors 200 Exploring each method of the descriptor protocol 203 The get method 203 The set method 205 The delete method 207 The set name method 210 Types of descriptors 211 Non-data descriptors 212 Data descriptors 214 Descriptors in action 217 An application of descriptors 217 A first attempt without using descriptors 217 The idiomatic implementation 219 Different forms of implementing descriptors 222 The issue of shared state 222 Accessing the dictionary of the object 223 t.me/DevTwitter
Page
12
Table of Contents [ v ] Using weak references 224 More considerations about descriptors 225 Reusing code 225 An alternative to class decorators 226 Analysis of descriptors 230 How Python uses descriptors internally 230 Functions and methods 230 Built-in decorators for methods 234 Slots 235 Implementing descriptors in decorators 237 Final remarks about descriptors 237 Interface of descriptors 237 Object-oriented design of the descriptors 238 Type annotations on descriptors 238 Summary 239 References 240 Chapter 7: Generators, Iterators, and Asynchronous Programming 241 Technical requirements 242 Creating generators 242 A first look at generators 242 Generator expressions 246 Iterating idiomatically 247 Idioms for iteration 247 The next() function 249 Using a generator 250 Itertools 251 Simplifying code through iterators 252 The iterator pattern in Python 255 Coroutines 259 The methods of the generator interface 260 close() 260 throw(ex_type[, ex_value[, ex_traceback]]) 261 send(value) 263 More advanced coroutines 266 Returning values in coroutines 266 Delegating into smaller coroutines – the 'yield from' syntax 268 Asynchronous programming 273 Magic asynchronous methods 275 Asynchronous context managers 276 Other magic methods 277 Asynchronous iteration 278 Asynchronous generators 280 Summary 281 t.me/DevTwitter
Page
13
Table of Contents [ vi ] References 282 Chapter 8: Unit Testing and Refactoring 285 Design principles and unit testing 286 A note about other forms of automated testing 287 Unit testing and agile software development 288 Unit testing and software design 289 Defining the boundaries of what to test 293 Tools for testing 293 Frameworks and libraries for unit testing 294 unittest 295 pytest 301 Code coverage 306 Mock objects 309 Refactoring 315 Evolving our code 316 Production code isn't the only one that evolves 318 More about testing 319 Property-based testing 320 Mutation testing 320 Common themes in testing 323 Boundaries or limit values 323 Classes of equivalence 323 Edge cases 324 A brief introduction to test-driven development 325 Summary 325 References 326 Chapter 9: Common Design Patterns 327 Design pattern considerations in Python 328 Design patterns in action 329 Creational patterns 330 Factories 330 Singleton and shared state (monostate) 331 Builder 336 Structural patterns 337 Adapter 337 Composite 339 Decorator 340 Facade 343 Behavioral patterns 344 Chain of responsibility 345 The template method 348 Command 349 State 350 t.me/DevTwitter
Page
14
Table of Contents [ vii ] The null object pattern 355 Final thoughts about design patterns 358 The influence of patterns over the design 358 Design patterns as theory 359 Names in our models 360 Summary 360 References 361 Chapter 10: Clean Architecture 363 From clean code to clean architecture 364 Separation of concerns 364 Monolithic applications and microservices 366 Abstractions 367 Software components 368 Packages 369 Managing dependencies 372 Other considerations when managing dependencies 374 Artifact versions 376 Docker containers 376 Use case 378 The code 379 Domain models 380 Calling from the application 382 Adapters 383 The services 384 Analysis 388 The dependency flow 388 Limitations 389 Testability 389 Intention revealing 390 Summary 390 References 391 Summing it all up 391 Share your experience 392 Other Books You May Enjoy 393 Index 395 t.me/DevTwitter
Page
15
t.me/DevTwitter
Page
16
[ ix ] Preface Who this book is for? This book is suitable for all software engineering practitioners who are interested in software design or learning more about Python. It is assumed that the reader is already familiar with the principles of object-oriented software design and has experience writing code. It will appeal to team leads, software architects and senior software engineers who want to learn good Python coding techniques to create projects from scratch or work on their legacy systems to save costs and improve efficiency. The book is organized in such a way that the content is in increasing order of complexity. The first chapters cover the basics of Python, which is a good way to learn the main idioms, functions, and utilities available in the language. The idea is not just to solve some problems with Python, but to do so in an idiomatic way. Experienced programmers will also benefit from the topics in this book, as some sections cover advanced topics in Python, such as decorators, descriptors, and an introduction to asynchronous programming. It will help the reader discover more about Python because some of the cases are analyzed from the internals of the language itself. Scientists using Python for data processing can also benefit from the content of this book, and to that end, several parts of the book are dedicated to setting up projects from the ground up, in terms of tools, configuration of environments, and good practices to release software. t.me/DevTwitter
Page
17
Preface [ x ] It is worth emphasizing the word "practitioners" in the first sentence of this section. This is a book that takes a pragmatic approach. Examples are limited to what the case study requires but are also intended to resemble the context of a real software project. It is not an academic book, and as such the definitions made, the remarks made, and the recommendations are to be taken with caution. The reader is expected to examine these recommendations critically and pragmatically rather than dogmatically. After all, practicality beats purity. What this book covers Chapter 1, Introduction, Code Formatting, and Tools, is an introduction to the main tools the reader needs to set up a development environment in Python. We cover the basics a Python developer is recommended to know to start working with the language effectively. It also includes some guidelines for maintaining readable code in the project, such as tools for static analysis, documentation, type checking, and code formatting. Having a common understanding of coding standards is a good thing but relying on good intentions only doesn't scale. That's why the chapter concludes by discussing tools to work more effectively. Chapter 2, Pythonic Code, looks at the first idioms in Python, which we will continue to use in the following chapters. We cover the particular features of Python, how they are meant to be used, and in this chapter, we start building knowledge around the idea that Pythonic code is in general of much better quality. Chapter 3, General Traits of Good Code, reviews general principles of software engineering making the focus on writing more maintainable code. With the knowledge gained from the previous chapter, we take a look at general clean design ideas, and how they can be implemented in Python. Chapter 4, The SOLID Principles, covers a set of design principles for object-oriented software design. This acronym is part of the language or jargon of software engineering, and we see how each one of them can be applied to Python. In particular, the reader will learn how dependency injection makes the code more maintainable, a concept that will be very useful in the next chapters. Chapter 5, Using Decorators to Improve Our Code, looks at one of the greatest features of Python. After understanding how to create decorators (for functions and classes), we put them in action for reusing code, separating responsibilities, and creating more granular functions. Another interesting learning point from this chapter is how to use decorators to our advantage to simplify complex and repetitive function signatures. t.me/DevTwitter
Page
18
Preface [ xi ] Chapter 6, Getting More Out of Our Objects with Descriptors, explores descriptors in Python, which take object-oriented design to a new level. While this is a feature more related to frameworks and tools, we can see how to improve the readability of our code with descriptors, and also reuse code. The content revisited in this chapter will make the reader achieve a higher level of understanding of Python. Chapter 7, Generators, Iterators, and Asynchronous Programming, starts by showing how generators are a fantastic feature of Python. The fact that iteration is a core component of Python could make us think that it leads to a new programming model. By using generators and iterators in general, we can think differently about the way we write our programs. With the lessons learned from generators, we go further and learn about coroutines in Python and the basics of asynchronous programming. This chapter wraps up by explaining the new syntax (and new magic methods!) for asynchronous programming and asynchronous iteration. Chapter 8, Unit Testing and Refactoring, discusses the importance of unit tests in any codebase that claims to be maintainable. We discuss refactoring as a pre-requisite to evolve and maintain a code base, and how unit tests are critical for this. All of this, with the support of the proper tools (mainly the unittest and pytest modules). Finally, we learn how the secret for good testing lies not so much on the tests themselves, but on having testable code. Chapter 9, Common Design Patterns, reviews how to implement the most common design patterns in Python, not from the point of view of solving a problem, but by examining how they solve problems by leveraging a better and more maintainable solution. The chapter mentions the peculiarities of Python that have made some of the design patterns invisible and takes a pragmatic approach to implement some of them. We discuss other (not so "conventional") patterns that are Python-specific. Chapter 10, Clean Architecture, focuses on the idea that clean code is the base of good architecture. All those details we mentioned in the first chapter, and everything else revisited along the way, will play a critical role in the entire design when the system is deployed. To get the most out of this book The reader is expected to have some degree of programming experience and be somehow familiarized with the basics of Python's syntax. Also, basic programming knowledge, like structured programming, and knowledge of object-oriented design is assumed. To test the code, you need to have Python installed, which can be downloaded from https://www.python.org/downloads/. The code runs with Python 3.9+, and the creation of a virtual environment is highly recommended. Alternative, the code can be tested in a Docker image. t.me/DevTwitter
Page
19
Preface [ xii ] Download the example code files The code bundle for the book is hosted on GitHub at https://github.com/ PacktPublishing/Clean-Code-in-Python-Second-Edition. We also have other code bundles from our rich catalog of books and videos available at https://github.com/ PacktPublishing/. Check them out! Download the color images We also provide a PDF file that has color images of the screenshots/diagrams used in this book. You can download it here: https://static.packt-cdn.com/ downloads/9781800560215_ColorImages.pdf Conventions used There are a number of text conventions used throughout this book. CodeInText: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. For example; "Then, just running the pylint command is enough to check it in the code." A block of code is set as follows: @dataclass class Point: lat: float long: float When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold: setup( name="apptool", description="Description of the intention of the package", long_description=long_description, ) t.me/DevTwitter
Page
20
Preface [ xiii ] Any command-line input or output is written as follows: >>> locate.__annotations__ {'latitude': float, 'longitue': float, 'return': __main__.Point} Bold: Indicates a new term, an important word, or words that you see on the screen, for example, in menus or dialog boxes, also appear in the text like this. For example: "We would like a design with a better separation of concerns." Get in touch Feedback from our readers is always welcome. General feedback: Email feedback@packtpub.com and mention the book's title in the subject of your message. If you have questions about any aspect of this book, please email us at questions@packtpub.com. Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit, http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details. Piracy: If you come across any illegal copies of our works in any form on the Internet, we would be grateful if you would provide us with the location address or website name. Please contact us at copyright@packtpub.com with a link to the material. If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit http://authors.packtpub.com. Warnings or important notes appear like this. Tips and tricks appear like this. t.me/DevTwitter
The above is a preview of the first 20 pages. Register to read the complete e-book.
AI Reading Assistant
Whole-book reading guide from stratified index samples; jump to passages in the text
AI guide
【One-Line Pitch】
A practical field guide to writing maintainable Python: it treats clean code as an engineering necessity rather than a luxury, showing how formatting, documentation, Pythonic idioms, and disciplined design keep technical debt from sinking a project. Best for intermediate Python developers and team leads who want habits and tooling they can apply immediately.
【Book Arc】
- **Opening (~0%–12%)**: Defines what clean code means and why it matters, then moves into formatting, documentation (docstrings, annotations, comments), and automated tooling—type checkers, linters, and auto-formatters like black—so quality checks run as part of the build rather than by willpower.
- **Early (~12%–32%)**: Builds "Pythonic" fluency: indexes and slices, custom sequences, context managers (including the `contextlib.contextmanager` decorator), comprehensions and assignment expressions, underscore conventions, data classes, and the pitfalls of extending built-in types.
- **Middle (~32%–50%)**: Shifts from idiom to design—general traits of good code, contracts and preconditions, defensive handling of input data, and exception-handling discipline (avoiding silent failures, using `contextlib.suppress`, chaining with `raise ... from`). Also warns against mixing implementation objects with domain objects, e.g. subclassing `dict` for a policy class.
- **Late (percentages beyond ~50% are not covered by the excerpts)**: The excerpts stop mid-discussion of object-oriented design concerns; later chapters on advanced design patterns, testing, and refactoring are not represented here.
- **Ending**: Not covered by the excerpts.
【Book Arc note】The progression is clear where the excerpts reach: mindset and tooling → language idioms → design and error-handling principles. Treat the second half as unverified territory.
【Key Takeaways】
- **Clean code is a necessity, not a luxury** (Opening): the opening frames quality as protection against accumulating technical debt, which is why formatting and documentation get first-class treatment rather than being dismissed as busywork.
- **Automate your standards** (Opening): agreeing on a style guide is only step one; the book stresses enforcing it through configuration files and tools (pylint, Coala, black) that run automatically, and even writing custom rules for recurring error-prone patterns.
- **Document intent, not mechanics** (Opening): docstrings and annotations are presented as Python's real documentation tools, distinct from comments; annotations gain further value in design, such as declaring interfaces for dependency injection.
- **Pythonic idioms reduce cognitive load** (Early): context managers, comprehensions, and data classes are framed as ways to make code express intent directly—e.g. `coord in grid` reading as a statement of the problem—so the whole team can focus on the essence rather than structure.
- **Respect the underscore convention** (Early): since Python attributes are public by default, the single-underscore prefix is a contract; keeping non-interface members private preserves refactoring freedom and avoids ripple effects.
- **Don't extend built-in types naively** (Early): subclassing `list` or `dict` directly produces surprising behavior in CPython because methods don't call each other; the `collections` module (`UserDict`, etc.) is the robust route.
- **Fail loudly and preserve context** (Middle): empty `except` blocks hide intent—use `contextlib.suppress` for deliberate ignores—and when wrapping exceptions, chain the original with `raise ... from` so the root cause survives.
- **Design objects by role, not convenience** (Middle): the transactional-policy example shows how inheriting from `dict` bloats the public interface and couples domain objects to implementation details; composition and delegation keep classes cohesive.
【Reading Tips】
- **Deep-read Chapters 1–2 if you're establishing team standards**: the tooling and idiom material is the most immediately actionable; set up black and a linter alongside your reading.
- **Skim the annotation/docstring section if you already use type hints**, but slow down on the "do annotations replace docstrings?" discussion—it clarifies a common confusion.
- **Treat the exception-handling and design chapters as a checklist**: after reading, audit your codebase for bare `except: pass`, unchained re-raises, and classes that subclass containers for convenience.
- **Watch for the recurring theme rather than memorizing rules**: nearly every example contrasts a tempting shortcut with a slightly more verbose but intention-revealing alternative; internalize that trade-off pattern.
- **Verify later chapters independently**: since the excerpts don't cover the book's second half, don't assume coverage of testing or refactoring based on this guide.
【Coverage Limits】
This guide is based on stratified excerpts covering roughly the first half of the book (through object-oriented design discussion); later chapters on advanced patterns, testing, and refactoring are not represented, so claims about the book's second half should be checked against the table of contents or the book itself.
Passage locations
Excerpt 1
elp from automated tools that will ease our work. For this reason, we'll discuss how to configure tools that will automatically run on the project as part of...
View in text
Excerpt 2
lp them focus on the true essence of the problem, and will help them avoid making mistakes. The goals of this chapter are as follows: • To understand indices...
View in text
Excerpt 3
bute) as a parameter. [ 59 ] t.me/DevTwitter Pythonic Code age = user_metadata.pop("age") return f"{name} ({age})" Let's conclude the section by understandin...
View in text
Excerpt 4
nt.name, score_for_student(student) ) ) A fair disclaimer: This is just an analysis of one of the traits of code duplication. In reality, there are more case...
View in text
Recommended for You
{{#thumbnailUrl}}
{{/thumbnailUrl}}
{{^thumbnailUrl}}
{{/thumbnailUrl}}
Loading recommended books...
Failed to load, please try again later
Tip the Site
Scan the WeChat Pay or Alipay code to tip. No login required.
WeChat Pay
Alipay