Python Concurrency with asyncio (Matthew Fowler)（Z-Library）

Page 1

M A N N I N G Matthew Fowler

Page 2

Continued on inside back cover I want to . . . How? Chapter(s) Learn the basics of single-threaded concurrency Understand how selectors and the event loop work 1, 3 Run two operations concurrently Use asyncio tasks and API functions 2, 4 Add a timeout to a long-running task Use the task’s wait_for method 2 Cancel a long-running task Use the task’s cancel method 2 Learn how non-blocking sockets work Create a non-blocking echo server with selectors 3 Handle graceful shutdowns of asyncio applications Use the signals API (*nix systems only) 3 Run multiple tasks concurrently Use asyncio API functions such as gather, wait, and as_completed 4 Run asynchronous web API requests Use a library, such as aiohttp, with asyncio API functions 4 Run multiple SQL queries concurrently Use an asynchronous library like asyncpg with asyncio API functions and connection pools 5 Run CPU-intensive work concurrently Use multiprocessing and process pools 6 Use shared state among multiple processes Save the state to shared memory 6 Avoid race conditions in multiprocessing code Use multiprocessing locks 6 Run blocking I/O-based APIs, such as requests, concurrently Use multithreading and thread pools 7 Avoid race conditions and deadlocks in multi- threading code Use multithreading locks and reentrant locks 7 Build a responsive GUI with asyncio Use multithreading with threading queues 7 Run multiple CPU-intensive tasks, such as data analysis with NumPy, concurrently Use multiprocessing, multithreading in certain circumstances 6, 7 Build a non-blocking command line application Use streams to asynchronously read data (*nix systems only) 8 Build a web application with asyncio Use a web framework with ASGI support such as Starlette or Django 9 Use WebSockets asynchronously Use the WebSocket library 9 Build a resilient backend-for-frontend microservice architecture with async concepts Use asyncio API functions with retries and the circuit breaker pattern 10 Prevent single-threaded race conditions Use asyncio locks 11 Limit the number of tasks running concurrently Use asyncio semaphores 11 Wait until an event occurs before performing an operation Use asyncio events 11 Control access to a shared resource Use asyncio condtions 11

Page 3

Python Concurrency with asyncio MATTHEW FOWLER MANN I NG SHELTER ISLAND

Page 4

For online information and ordering of this and other Manning books, please visit www.manning.com. The publisher offers discounts on this book when ordered in quantity. For more information, please contact Special Sales Department Manning Publications Co. 20 Baldwin Road PO Box 761 Shelter Island, NY 11964 Email: orders@manning.com ©2022 by Manning Publications Co. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by means electronic, mechanical, photocopying, or otherwise, without prior written permission of the publisher. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in the book, and Manning Publications was aware of a trademark claim, the designations have been printed in initial caps or all caps. Recognizing the importance of preserving what has been written, it is Manning’s policy to have the books we publish printed on acid-free paper, and we exert our best efforts to that end. Recognizing also our responsibility to conserve the resources of our planet, Manning books are printed on paper that is at least 15 percent recycled and processed without the use of elemental chlorine. The author and publisher have made every effort to ensure that the information in this book was correct at press time. The author and publisher do not assume and hereby disclaim any liability to any party for any loss, damage, or disruption caused by errors or omissions, whether such errors or omissions result from negligence, accident, or any other cause, or from any usage of the information herein. Manning Publications Co. Development editor: Doug Rudder 20 Baldwin Road Technical development editor: Robert Wenner PO Box 761 Review editor: Mihaela Batinić Shelter Island, NY 11964 Production editor: Andy Marinkovich Copy editor: Christian Berk Proofreader: Keri Hales Technical proofreader: Mayur Patil Typesetter: Dennis Dalinnik Cover designer: Marija Tudor ISBN: 9781617298660 Printed in the United States of America

Page 5

To my beautiful wife Kathy, thank you for always being there.

Page 6

(This page has no text content)

Page 7

contents preface xi acknowledgments xiii about this book xiv about the author xvii about the cover illustration xviii 1 Getting to know asyncio 1 1.1 What is asyncio? 2 1.2 What is I/O-bound and what is CPU-bound? 3 1.3 Understanding concurrency, parallelism, and multitasking 4 Concurrency 4 ■ Parallelism 5 ■ The difference between concurrency and parallelism 6 ■ What is multitasking? 7 The benefits of cooperative multitasking 7 1.4 Understanding processes, threads, multithreading, and multiprocessing 8 Process 8 ■ Thread 8 1.5 Understanding the global interpreter lock 12 Is the GIL ever released? 15 ■ asyncio and the GIL 17v

Page 8

CONTENTSvi1.6 How single-threaded concurrency works 17 What is a socket? 17 1.7 How an event loop works 20 2 asyncio basics 23 2.1 Introducing coroutines 24 Creating coroutines with the async keyword 24 ■ Pausing execution with the await keyword 26 2.2 Introducing long-running coroutines with sleep 27 2.3 Running concurrently with tasks 30 The basics of creating tasks 30 ■ Running multiple tasks concurrently 31 2.4 Canceling tasks and setting timeouts 33 Canceling tasks 34 ■ Setting a timeout and canceling with wait_for 35 2.5 Tasks, coroutines, futures, and awaitables 37 Introducing futures 37 ■ The relationship between futures, tasks, and coroutines 39 2.6 Measuring coroutine execution time with decorators 40 2.7 The pitfalls of coroutines and tasks 42 Running CPU-bound code 42 ■ Running blocking APIs 44 2.8 Accessing and manually managing the event loop 45 Creating an event loop manually 46 ■ Accessing the event loop 46 2.9 Using debug mode 47 Using asyncio.run 47 ■ Using command-line arguments 47 Using environment variables 48 3 A first asyncio application 50 3.1 Working with blocking sockets 51 3.2 Connecting to a server with Telnet 53 Reading and writing data to and from a socket 54 ■ Allowing multiple connections and the dangers of blocking 56 3.3 Working with non-blocking sockets 57 3.4 Using the selectors module to build a socket event loop 61

Page 9

CONTENTS vii3.5 An echo server on the asyncio event loop 64 Event loop coroutines for sockets 64 ■ Designing an asyncio echo server 65 ■ Handling errors in tasks 67 3.6 Shutting down gracefully 69 Listening for signals 69 ■ Waiting for pending tasks to finish 70 4 Concurrent web requests 75 4.1 Introducing aiohttp 76 4.2 Asynchronous context managers 77 Making a web request with aiohttp 79 ■ Setting timeouts with aiohttp 81 4.3 Running tasks concurrently, revisited 82 4.4 Running requests concurrently with gather 84 Handling exceptions with gather 86 4.5 Processing requests as they complete 88 Timeouts with as_completed 90 4.6 Finer-grained control with wait 92 Waiting for all tasks to complete 92 ■ Watching for exceptions 94 ■ Processing results as they complete 96 Handling timeouts 99 ■ Why wrap everything in a task? 100 5 Non-blocking database drivers 102 5.1 Introducing asyncpg 103 5.2 Connecting to a Postgres database 103 5.3 Defining a database schema 104 5.4 Executing queries with asyncpg 107 5.5 Executing queries concurrently with connection pools 109 Inserting random SKUs into the product database 110 Creating a connection pool to run queries concurrently 113 5.6 Managing transactions with asyncpg 118 Nested transactions 119 ■ Manually managing transactions 120 5.7 Asynchronous generators and streaming result sets 122 Introducing asynchronous generators 123 ■ Using asynchronous generators with a streaming cursor 124

Page 10

CONTENTSviii6 Handling CPU-bound work 128 6.1 Introducing the multiprocessing library 129 6.2 Using process pools 131 Using asynchronous results 132 6.3 Using process pool executors with asyncio 133 Introducing process pool executors 133 ■ Process pool executors with the asyncio event loop 134 6.4 Solving a problem with MapReduce using asyncio 136 A simple MapReduce example 137 ■ The Google Books Ngram dataset 139 ■ Mapping and reducing with asyncio 140 6.5 Shared data and locks 145 Sharing data and race conditions 146 ■ Synchronizing with locks 149 ■ Sharing data with process pools 151 6.6 Multiple processes, multiple event loops 154 7 Handling blocking work with threads 159 7.1 Introducing the threading module 160 7.2 Using threads with asyncio 164 Introducing the requests library 164 ■ Introducing thread pool executors 165 ■ Thread pool executors with asyncio 167 Default executors 168 7.3 Locks, shared data, and deadlocks 169 Reentrant locks 171 ■ Deadlocks 173 7.4 Event loops in separate threads 175 Introducing Tkinter 176 ■ Building a responsive UI with asyncio and threads 178 7.5 Using threads for CPU-bound work 185 Multithreading with hashlib 185 ■ Multithreading with NumPy 188 8 Streams 191 8.1 Introducing streams 192 8.2 Transports and protocols 192 8.3 Stream readers and stream writers 196 8.4 Non-blocking command-line input 198 Terminal raw mode and the read coroutine 202

Page 11

CONTENTS ix8.5 Creating servers 209 8.6 Creating a chat server and client 211 9 Web applications 217 9.1 Creating a REST API with aiohttp 218 What is REST? 218 ■ aiohttp server basics 219 ■ Connecting to a database and returning results 220 ■ Comparing aiohttp with Flask 226 9.2 The asynchronous server gateway interface 228 How does ASGI compare to WSGI? 228 9.3 ASGI with Starlette 230 A REST endpoint with Starlette 230 ■ WebSockets with Starlette 231 9.4 Django asynchronous views 235 Running blocking work in an asynchronous view 240 Using async code in synchronous views 242 10 Microservices 244 10.1 Why microservices? 245 Complexity of code 245 ■ Scalability 246 ■ Team and stack independence 246 ■ How can asyncio help? 246 10.2 Introducing the backend-for-frontend pattern 246 10.3 Implementing the product listing API 248 User favorite service 248 ■ Implementing the base services 249 Implementing the backend-for-frontend service 253 ■ Retrying failed requests 258 ■ The circuit breaker pattern 261 11 Synchronization 267 11.1 Understanding single-threaded concurrency bugs 268 11.2 Locks 272 11.3 Limiting concurrency with semaphores 276 Bounded semaphores 278 11.4 Notifying tasks with events 280 11.5 Conditions 285 12 Asynchronous queues 290 12.1 Asynchronous queue basics 291 Queues in web applications 297 ■ A web crawler queue 300

Page 12

CONTENTSx12.2 Priority queues 303 12.3 LIFO queues 309 13 Managing subprocesses 312 13.1 Creating a subprocess 313 Controlling standard output 315 ■ Running subprocesses concurrently 318 13.2 Communicating with subprocesses 322 14 Advanced asyncio 327 14.1 APIs with coroutines and functions 328 14.2 Context variables 330 14.3 Forcing an event loop iteration 331 14.4 Using different event loop implementations 333 14.5 Creating a custom event loop 334 Coroutines and generators 335 ■ Generator-based coroutines are deprecated 335 ■ Custom awaitables 337 ■ Using sockets with futures 340 ■ A task implementation 342 ■ Implementing an event loop 343 ■ Implementing a server with a custom event loop 346 index 349

Page 13

preface Nearly 20 years ago, I got my start in professional software engineering writing a mashup of Matlab, C++, and VB.net code to control and analyze data from mass spec- trometers and other laboratory devices. The thrill of seeing a line of code trigger a machine to move how I wanted always stuck with me, and ever since then, I knew soft- ware engineering was the career for me. Over the years, I gradually moved toward API development and distributed systems, mainly focusing on Java and Scala, learning a lot of Python along the way. I got my start in Python around 2015, primarily by working on a machine learning pipeline that took sensor data and used it to make predictions—such as sleep track- ing, step count, sit-to-stand transitions, and similar activities—about the sensor’s wearer. At the time, this machine learning pipeline was slow to the point that it was becoming a customer issue. One of the ways I worked on alleviating the issue was utilizing con- currency. As I dug into the knowledge available for learning concurrent programming in Python, I found things hard to navigate and learn compared to what I was used to in the Java world. Why doesn’t multithreading work the same way that it would in Java? Does it make more sense to use multiprocessing? What about the newly introduced asyncio? What is the global interpreter lock, and why does it exist? There weren’t a lot of books on the topic of concurrency in Python, and most knowledge was scattered throughout documentation and a smattering of blogs with varying consistency of qual- ity. Fast-forward to today, and things haven’t changed much. While there are more resources, the landscape is still sparse, disjointed, and not as friendly for newcomers to concurrency as it should be.xi

Page 14

PREFACExii Of course, a lot has changed in the past several years. Back then, asyncio was in its infancy and has since become an important module in Python. Now, single-threaded concurrency models and coroutines are a core component of concurrency in Python, in addition to multithreading and multiprocessing. This means the concurrency land- scape in Python has gotten larger and more complex, while still not having compre- hensive resources for those wanting to learn it. My motivation for writing this book was to fill this gap that exists in the Python landscape on the topic of concurrency, specifically with asyncio and single-threaded concurrency. I wanted to make the complex and under-documented topic of single- threaded concurrency more accessible to developers of all skill levels. I also wanted to write a book that would enhance generic understanding of concurrency topics outside of Python. Frameworks such as Node.js and languages such as Kotlin have single- threaded concurrency models and coroutines, so knowledge gained here is helpful in those domains as well. My hope is that all who read it find this book useful in their day-to-day lives as developers—not only within the Python landscape but also within the domain of concurrent programming.

Page 15

acknowledgments First, I want to thank my wife, Kathy, who was always there for me to proofread when I wasn’t sure if something made sense, and who was extremely supportive through the entire process. A close second goes to my dog, Dug, who was always around to drop his ball near me to remind me to take a break from writing to play. Next, I’d like to thank my editor, Doug Rudder, and my technical reviewer, Robert Wenner. Your feedback was invaluable in helping keep this book on schedule and high quality, ensuring that my code and explanations made sense and were easy to understand. To all the reviewers: Alexey Vyskubov, Andy Miles, Charles M. Shelton, Chris Viner, Christopher Kottmyer, Clifford Thurber, Dan Sheikh, David Cabrero, Didier Garcia, Dimitrios Kouzis-Loukas, Eli Mayost, Gary Bake, Gonzalo Gabriel Jiménez Fuentes, Gregory A. Lussier, James Liu, Jeremy Chen, Kent R. Spillner, Lakshmi Narayanan Narasimhan, Leonardo Taccari, Matthias Busch, Pavel Filatov, Phillip Sorensen, Richard Vaughan, Sanjeev Kilarapu, Simeon Leyzerzon, Simon Tschöke, Simone Sguazza, Sumit K. Singh, Viron Dadala, William Jamir Silva, and Zoheb Ainapore, your suggestions helped make this a better book. Finally, I want to thank the countless number of teachers, coworkers, and mentors I’ve had over the past years. I’ve learned and grown so much from all of you. The sum of the experiences we’ve had together has given me the tools needed to produce this work as well as succeed in my career. Without all of you, I wouldn’t be where I am today. Thank you!xiii

Page 16

about this book Python Concurrency with asyncio was written to teach you how to utilize concurrency in Python to improve application performance, throughput, and responsiveness. We start by focusing on core concurrency topics, explaining how asyncio’s model of single- threaded concurrency works as well as how coroutines and async/await syntax works. We then transition into practical applications of concurrency, such as making multiple web requests or database queries concurrently, managing threads and processes, building web applications, and handling synchronization issues. Who should read this book? This book is for intermediate to advanced developers who are looking to better understand and utilize concurrency in their existing or new Python applications. One of the goals of this book is to explain complex concurrency topics in plain, easy-to- understand language. To that end, no prior experience with concurrency is needed, though of course, it is helpful. In this book we’ll cover a wide range of uses, from web- based APIs to command-line applications, so this book should be applicable to many problems you’ll need to solve as a developer.xiv

Page 17

ABOUT THIS BOOK xvHow this book is organized: A road map This book is organized into 14 chapters, covering gradually more advanced topics that build on what you’ve learned in previous chapters. Chapter 1 focuses on basic concurrency knowledge in Python. We learn what CPU-bound and I/O-bound work is and introduce how asyncio’s single-threaded concurrency model works. Chapter 2 focuses on the basics of asyncio coroutines and how to use async/ await syntax to build applications utilizing concurrency. Chapter 3 focuses on how non-blocking sockets and selectors work and how to build an echo server using asyncio. Chapter 4 focuses on how to make multiple web requests concurrently. Doing this, we’ll learn more about the core asyncio APIs for running coroutines concurrently. Chapter 5 focuses on how to make multiple database queries concurrently using connection pools. We’ll also learn about asynchronous context managers and asynchronous generators in the context of databases Chapter 6 focuses on multiprocessing, specifically how to utilize it with asyncio to handle CPU-intensive work. We’ll build a map/reduce application to demon- strate this. Chapter 7 focuses on multithreading, specifically how to utilize it with asyncio to handle blocking I/O. This is useful for libraries that don’t have native asyncio support but can still benefit from concurrency. Chapter 8 focuses on network streams and protocols. We’ll use this to create a chat server and client capable of handling multiple users concurrently. Chapter 9 focuses on asyncio-powered web applications and the ASGI (asyn- chronous server gateway interface). We’ll explore a few ASGI frameworks and discuss how to build web APIs with them. We’ll also explore WebSockets. Chapter 10 describes how to use asyncio-based web APIs to build a hypothetical microservice architecture. Chapter 11 focuses on single-threaded concurrency synchronization issues and how to resolve them. We dive into locks, semaphores, events, and conditions. Chapter 12 focuses on asynchronous queues. We’ll use these to build a web appli- cation that responds to client requests instantly, despite doing time-consuming work in the background. Chapter 13 focuses on creating and managing subprocesses, showing you how to read from and write data to them. Chapter 14 focuses on advanced topics, such as forcing event loop iterations, context variables, and creating your own event loop. This information will be most useful to asyncio API designers and those interested in how the innards of the asyncio event loop function.

Page 18

ABOUT THIS BOOKxviAt minimum, you should read the first four chapters to get a full understanding of how asyncio works, how to build your first real application, and how to use the core asyncio APIs to run coroutines concurrently (covered in chapter 4). After this you should feel free to move around the book based on your interests. About the code This book contains many code examples, both in numbered listings and in-line. Some code listings are reused as imports in later listings in the same chapter, and some are reused across multiple chapters. Code reused across multiple chapters will assume you’ve created a module named util; you’ll create this in chapter 2. For each individual code listing, we will assume you have created a module for that chapter named chapter_ {chapter_number} and then put the code in a file of the format listing_{chapter_ number}_{listing_number}.py within that module. For example, the code for listing 2.2 in chapter 2 will be in a module called chapter_2 in a file named listing_2_2.py. Several places in the book go through performance numbers, such as time for a program to complete or web requests completed per second. Code samples in this book were run and benchmarked on a 2019 MacBook Pro with a 2.4 GHz 8-Core Intel Core i9 processor and 32 GB 2667 MHz DDR4 RAM, using a gigabit wireless internet connection. Depending on the machine you run on, these numbers will be different, and factors of speedup or improvement will be different. Executable snippets of code can be found in the liveBook (online) version of this book at https://livebook.manning.com/book/python-concurrency-with-asyncio. The complete source code can be downloaded free of charge from the Manning website at https://www.manning.com/books/python-concurrency-with-asyncio, and is also avail- able on Github at https://github.com/concurrency-in-python-with-asyncio. liveBook discussion forum Purchase of Python Concurrency with asyncio includes free access to liveBook, Manning’s online reading platform. Using liveBook’s exclusive discussion features, you can attach comments to the book globally or to specific sections or paragraphs. It’s a snap to make notes for yourself, ask and answer technical questions, and receive help from the author and other users. To access the forum, go to https://livebook.manning .com/#!/book/python-concurrency-with-asyncio/discussion. You can also learn more about Manning’s forums and the rules of conduct at https://livebook.manning.com/ #!/discussion. Manning’s commitment to our readers is to provide a venue where a meaningful dialogue between individual readers and between readers and the author can take place. It is not a commitment to any specific amount of participation on the part of the author, whose contribution to the forum remains voluntary (and unpaid). We sug- gest you try asking the author some challenging questions lest his interest stray! The forum and the archives of previous discussions will be accessible from the publisher’s website for as long as the book is in print.

Page 19

about the author MATTHEW FOWLER has nearly 20 years of software engineering experience in roles from software architect to engineering director. He started out writing software for scientific applica- tions and moved into full-stack web development and distrib- uted systems, eventually leading multiple teams of developers and managers to do the same for an e-commerce site with tens of millions of users. He lives in Lexington, Massachusetts with his wife, Kathy.xvii

Page 20

about the cover illustration The figure on the cover of Python Concurrency with asyncio is “Paysanne du Marquisat de Bade,” or Peasant woman of the Marquisate of Baden, taken from a book by Jacques Grasset de Saint-Sauveur published in 1797. Each illustration is finely drawn and col- ored by hand. In those days, it was easy to identify where people lived and what their trade or sta- tion in life was just by their dress. Manning celebrates the inventiveness and initiative of the computer business with book covers based on the rich diversity of regional cul- ture centuries ago, brought back to life by pictures from collections such as this one.xviii