Microservice APIs Using Python, Flask, FastAPI, OpenAPI and more (Jose Haro Peralta) (z-library.sk, 1lib.sk, z-lib.sk)

M A N N I N G José Haro Peralta Using Python, Flask, FastAPI, OpenAPI and more

Documentation-driven development is an API-first development method in which you design and document the API first; then, you build the API server and the API client against the documentation; and finally, you use the API documentation to validate the server and client implementations. Documentation-driven development helps you reduce the chances of API integration failure, and it gives you more control and visibility of integration errors. 3. Test the implementation against the specification. API specification API server developers API client developers 1. API design and documentation 2. Build the client and the server against the API documentation. REST APIs are structured around endpoints. We distinguish between singleton endpoints, such as GET /orders/8, and collection endpoints, such as GET /orders. REST APIs leverage the semantics of HTTP methods to indicate actions (such as POST to create resources), and they use HTTP status codes that signal the result of processing the request (such as 200 for successful responses). Singleton endpoints PUT /orders/8 {payload} DELETE /orders/8 Update an order Delete an order Status code: 200 {payload} Status code: 204HTTP equestsr HTTP esponsesr POST /orders/8/pay {payload} POST /orders/8/cancel {payload} Pay for an order Cancel an order Status code: 200 {payload} Status code: 200 {payload} Collection endpoints POST /orders {payload} Place an order Status code: 201 {payload} GET /orders Get a collection of orders Status code: 200 {payload} GET /orders/8 Retrieve an order Status code: 200 {payload} Singleton endpoints Orders service API lientc

Microservice APIs

Microservice APIs USING PYTHON, FLASK, FASTAPI, OPENAPI AND MORE JOSÉ HARO PERALTA MANN I NG SHELTER ISLAND

For online information and ordering of this and other Manning books, please visit www.manning.com. The publisher offers discounts on this book when ordered in quantity. For more information, please contact Special Sales Department Manning Publications Co. 20 Baldwin Road PO Box 761 Shelter Island, NY 11964 Email: orders@manning.com ©2023 by Manning Publications Co. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by means electronic, mechanical, photocopying, or otherwise, without prior written permission of the publisher. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in the book, and Manning Publications was aware of a trademark claim, the designations have been printed in initial caps or all caps. Recognizing the importance of preserving what has been written, it is Manning’s policy to have the books we publish printed on acid-free paper, and we exert our best efforts to that end. Recognizing also our responsibility to conserve the resources of our planet, Manning books are printed on paper that is at least 15 percent recycled and processed without the use of elemental chlorine. The author and publisher have made every effort to ensure that the information in this book was correct at press time. The author and publisher do not assume and hereby disclaim any liability to any party for any loss, damage, or disruption caused by errors or omissions, whether such errors or omissions result from negligence, accident, or any other cause, or from any usage of the information herein. Manning Publications Co. Development editor: Marina Michaels 20 Baldwin Road Technical development editor: Nick Watts PO Box 761 Review editor: Mihaela Batinić Shelter Island, NY 11964 Production editor: Andy Marinkovich Copy editor: Michele Mitchell Proofreader: Katie Tennant Technical proofreader: Al Krinker Typesetter: Dennis Dalinnik Cover designer: Marija Tudor ISBN: 9781617298417 Printed in the United States of America

To Jiwon, without whose constant support and encouragement I wouldn’t have been able to write this book, and to Ivy, that boundless spark of joy that makes everything I do worth it.

brief contents PART 1 INTRODUCING MICROSERVICE APIS...............................1 1 ■ What are microservice APIs? 3 2 ■ A basic API implementation 20 3 ■ Designing microservices 45 PART 2 DESIGNING AND BUILDING REST APIS ........................59 4 ■ Principles of REST API design 61 5 ■ Documenting REST APIs with OpenAPI 90 6 ■ Building REST APIs with Python 110 7 ■ Service implementation patterns for microservices 144 PART 3 DESIGNING AND BUILDING GRAPHQL APIS . ..............183 8 ■ Designing GraphQL APIs 185 9 ■ Consuming GraphQL APIs 210 10 ■ Building GraphQL APIs with Python 233vii

BRIEF CONTENTSviiiPART 4 SECURING, TESTING, AND DEPLOYING MICROSERVICE APIS ..................................................267 11 ■ API authorization and authentication 269 12 ■ Testing and validating APIs 302 13 ■ Dockerizing microservice APIs 331 14 ■ Deploying microservice APIs with Kubernetes 342

contents preface xvi acknowledgments xviii about this book xx about the author xxv about the cover illustration xxvi PART 1 INTRODUCING MICROSERVICE APIS.....................1 1 What are microservice APIs? 3 1.1 What are microservices? 4 Defining microservices 4 ■ Microservices vs. monoliths 5 Microservices today and how we got here 7 1.2 What are web APIs? 8 What is an API? 8 ■ What is a web API? 9 ■ How do APIs help us drive microservices integrations? 9 1.3 Challenges of microservices architecture 11 Effective service decomposition 11 ■ Microservices integration tests 12 ■ Handling service unavailability 12 ■ Tracing distributed transactions 13 ■ Increased operational complexity and infrastructure overhead 14 1.4 Introducing documentation-driven development 15ix

CONTENTSx1.5 Introducing the CoffeeMesh application 17 1.6 Who this book is for and what you will learn 17 2 A basic API implementation 20 2.1 Introducing the orders API specification 21 2.2 High-level architecture of the orders application 22 2.3 Implementing the API endpoints 23 2.4 Implementing data validation models with pydantic 30 2.5 Validating request payloads with pydantic 34 2.6 Marshalling and validating response payloads with pydantic 38 2.7 Adding an in-memory list of orders to the API 41 3 Designing microservices 45 3.1 Introducing CoffeeMesh 46 3.2 Microservices design principles 46 Database-per-service principle 46 ■ Loose coupling principle 48 Single Responsibility Principle 49 3.3 Service decomposition by business capability 49 Analyzing the business structure of CoffeeMesh 49 ■ Decomposing microservices by business capabilities 50 3.4 Service decomposition by subdomains 52 What is domain-driven design? 52 ■ Applying strategic analysis to CoffeeMesh 53 3.5 Decomposition by business capability vs. decomposition by subdomain 57 PART 2 DESIGNING AND BUILDING REST APIS ..............59 4 Principles of REST API design 61 4.1 What is REST? 62 4.2 Architectural constraints of REST applications 63 Separation of concerns: The client-server architecture principle 64 Make it scalable: The statelessness principle 64 ■ Optimize for performance: The cacheability principle 65 ■ Make it simple for the client: The layered system principle 66 ■ Extendable interfaces: The code-on-demand principle 66 ■ Keep it consistent: The uniform interface principle 67

CONTENTS xi4.3 Hypermedia as the engine of application state 67 4.4 Analyzing the maturity of an API with the Richardson maturity model 70 Level 0: Web APIs à la RPC 71 ■ Level 1: Introducing the concept of resource 71 ■ Level 2: Using HTTP methods and status codes 72 ■ Level 3: API discoverability 72 4.5 Structured resource URLs with HTTP methods 73 4.6 Using HTTP status codes to create expressive HTTP responses 77 What are HTTP status codes? 77 ■ Using HTTP status codes to report client errors in the request 78 ■ Using HTTP status codes to report errors in the server 82 4.7 Designing API payloads 83 What are HTTP payloads, and when do we use them? 83 ■ HTTP payload design patterns 84 4.8 Designing URL query parameters 87 5 Documenting REST APIs with OpenAPI 90 5.1 Using JSON Schema to model data 91 5.2 Anatomy of an OpenAPI specification 95 5.3 Documenting the API endpoints 96 5.4 Documenting URL query parameters 97 5.5 Documenting request payloads 98 5.6 Refactoring schema definitions to avoid repetition 100 5.7 Documenting API responses 102 5.8 Creating generic responses 105 5.9 Defining the authentication scheme of the API 107 6 Building REST APIs with Python 110 6.1 Overview of the orders API 111 6.2 URL query parameters for the orders API 112 6.3 Validating payloads with unknown fields 115 6.4 Overriding FastAPI’s dynamically generated specification 118 6.5 Overview of the kitchen API 120 6.6 Introducing flask-smorest 122

CONTENTSxii6.7 Initializing the web application for the API 123 6.8 Implementing the API endpoints 125 6.9 Implementing payload validation models with marshmallow 129 6.10 Validating URL query parameters 133 6.11 Validating data before serializing the response 136 6.12 Implementing an in-memory list of schedules 140 6.13 Overriding flask-smorest’s dynamically generated API specification 142 7 Service implementation patterns for microservices 144 7.1 Hexagonal architectures for microservices 145 7.2 Setting up the environment and the project structure 148 7.3 Implementing the database models 149 7.4 Implementing the repository pattern for data access 155 The case for the repository pattern: What is it, and why is it useful? 155 ■ Implementing the repository pattern 157 7.5 Implementing the business layer 162 7.6 Implementing the unit of work pattern 172 7.7 Integrating the API layer and the service layer 177 PART 3 DESIGNING AND BUILDING GRAPHQL APIS......183 8 Designing GraphQL APIs 185 8.1 Introducing GraphQL 186 8.2 Introducing the products API 189 8.3 Introducing GraphQL’s type system 192 Creating property definitions with scalars 192 ■ Modeling resources with object types 193 ■ Creating custom scalars 194 8.4 Representing collections of items with lists 195 8.5 Think graphs: Building meaningful connections between object types 196 Connecting types through edge properties 196 ■ Creating connections with through types 198

CONTENTS xiii8.6 Combining different types through unions and interfaces 200 8.7 Constraining property values with enumerations 202 8.8 Defining queries to serve data from the API 203 8.9 Altering the state of the server with mutations 206 9 Consuming GraphQL APIs 210 9.1 Running a GraphQL mock server 211 9.2 Introducing GraphQL queries 214 Running simple queries 214 ■ Running queries with parameters 215 ■ Understanding query errors 215 9.3 Using fragments in queries 217 9.4 Running queries with input parameters 219 9.5 Navigating the API graph 219 9.6 Running multiple queries and query aliasing 221 Running multiple queries in the same request 221 ■ Aliasing our queries 222 9.7 Running GraphQL mutations 225 9.8 Running parameterized queries and mutations 226 9.9 Demystifying GraphQL queries 229 9.10 Calling a GraphQL API with Python code 230 10 Building GraphQL APIs with Python 233 10.1 Analyzing the API requirements 234 10.2 Introducing the tech stack 234 10.3 Introducing Ariadne 235 10.4 Implementing the products API 241 Laying out the project structure 241 ■ Creating an entry point for the GraphQL server 242 ■ Implementing query resolvers 243 Implementing type resolvers 246 ■ Handling query parameters 252 Implementing mutation resolvers 256 ■ Building resolvers for custom scalar types 258 ■ Implementing field resolvers 262

CONTENTSxivPART 4 SECURING, TESTING, AND DEPLOYING MICROSERVICE APIS ........................................267 11 API authorization and authentication 269 11.1 Setting up the environment for this chapter 270 11.2 Understanding authentication and authorization protocols 271 Understanding Open Authorization 271 ■ Understanding OpenID Connect 276 11.3 Working with JSON Web Tokens 278 Understanding the JWT header 279 ■ Understanding JWT claims 280 ■ Producing JWTs 282 ■ Inspecting JWTs 284 Validating JWTs 286 11.4 Adding authorization to the API server 287 Creating an authorization module 287 ■ Creating an authorization middleware 289 ■ Adding CORS middleware 292 11.5 Authorizing resource access 293 Updating the database to link users and orders 294 Restricting user access to their own resources 297 12 Testing and validating APIs 302 12.1 Setting up the environment for API testing 303 12.2 Testing REST APIs with Dredd 304 What is Dredd? 304 ■ Installing and running Dredd’s default test suite 305 ■ Customizing Dredd’s test suite with hooks 307 Using Dredd in your API testing strategy 315 12.3 Introduction to property-based testing 315 What is property-based testing? 315 ■ The traditional approach to API testing 316 ■ Property-based testing with Hypothesis 318 Using Hypothesis to test a REST API endpoint 319 12.4 Testing REST APIs with Schemathesis 322 Running Schemathesis’s default test suite 322 ■ Using links to enhance Schemathesis’ test suite 323 12.5 Testing GraphQL APIs 327 Testing GraphQL APIs with Schemathesis 327 12.6 Designing your API testing strategy 329

CONTENTS xv13 Dockerizing microservice APIs 331 13.1 Setting up the environment for this chapter 332 13.2 Dockerizing a microservice 333 13.3 Running applications with Docker Compose 338 13.4 Publishing Docker builds to a container registry 340 14 Deploying microservice APIs with Kubernetes 342 14.1 Setting up the environment for this chapter 343 14.2 How Kubernetes works: The “CliffsNotes” version 344 14.3 Creating a Kubernetes cluster with EKS 346 14.4 Using IAM roles for Kubernetes service accounts 350 14.5 Deploying a Kubernetes load balancer 351 14.6 Deploying microservices to the Kubernetes cluster 353 Creating a deployment object 354 ■ Creating a service object 357 Exposing services with ingress objects 359 14.7 Setting up a serverless database with AWS Aurora 361 Creating an Aurora Serverless database 361 ■ Managing secrets in Kubernetes 364 ■ Running the database migrations and connecting our service to the database 367 14.8 Updating the OpenAPI specification with the ALB’s hostname 370 14.9 Deleting the Kubernetes cluster 372 appendix A Types of web APIs and protocols 376 appendix B Managing an API’s life cycle 387 appendix C API authorization using an identity provider 391 index 403

preface APIs and microservices have taken the software industry by storm. Under the pressure of increasing software complexity and the need to scale, more and more organizations are migrating from monolithic to microservices architecture. O’Reilly’s “Microservices Adoption in 2020” report found that 77% of respondents had adopted microservices, a trend that is expected to continue growing in the coming years. Using microservices poses the challenge of driving service integrations through APIs. According to Nordic APIs, 90% of developers work with APIs and they spend 30% of their time building APIs.1 The growth of the API economy has transformed the way we build applications. Today, it’s more and more common to build products and services that are delivered entirely over APIs, such as Twilio and Stripe. Even tradi- tional sectors like banking and insurance are finding new lines of business by opening their APIs and integrating within the Open Banking ecosystem. The wide availability of API-first products means that we can focus on our core business capabilities when building our own applications, while using external APIs to handle common tasks such as authenticating users and sending emails. It’s exciting to be part of this growing ecosystem. However, before we embrace microservices and APIs, we need to know how to architect microservices, how to design APIs, how to define an API strategy, how to make sure we deliver reliable inte- grations, how to choose a deployment model, and how to protect our systems. In my 1 J. Simpson, “20 Impressive API Economy Statistics” (https://nordicapis.com/20-impressive-api-economy-statistics/ [accessed May 26, 2022]).xvi

PREFACE xviiexperience, most organizations struggle with one or more of these questions, and a recent report by IBM found that 31% of businesses haven’t adopted microservices due to lack of internal expertise.2 Equally, Postman’s 2022 State of the API Report found that 14% of respondents experience API integration failures 11%–25% of the time (http://mng.bz/Xa9v), and according to Salt Security, 94% of organizations experi- enced API security incidents in 2022.3 Many books address the problems mentioned in the previous paragraph, but they typically do it from a highly specific point of view: some focus on architecture, others on APIs, and yet others on security. I felt there’s a gap for a book that brings all these questions together and addresses them with a practical approach: essentially, a book that can get an average developer up and running quickly with the best practices, principles, and patterns for designing and building microservice APIs. I wrote this book with that goal in mind. Over the past years, I’ve had the opportunity to work with different clients helping them to architect microservices and deliver API integrations. Working on those proj- ects gave me a vantage view into the major hurdles that development teams face when working with microservices and APIs. As it turns out, both technologies are deceivingly simple. A well-designed API is easy to navigate and consume, while well-architected microservices boost developer productivity and are easily scalable. On the other side of the spectrum, badly designed APIs are error prone and difficult to use, and badly architected microservices result in so-called distributed monoliths. The obvious questions arise: How do you design good APIs? And how do you archi- tect loosely coupled microservices? This book will help you answer these questions and more. You’ll also get your hands dirty building APIs and services, and you’ll learn how to secure them, test them, and deploy them. The methods, patterns, and principles that I teach in this book are the outcome of many years of trials and experimentation, and I’m very excited about sharing them with you. I hope you find this book a valuable resource in your journey towards becoming a better software developer and architect. 2 “Microservices in the enterprise, 2021: Real benefits, worth the challenges,” (https://www.ibm.com/downloads/ cas/OQG4AJAM [accessed 26th May 2022]). 3 Salt Security, “State of API Security Q3 2022”, p. 4 (https://content.salt.security/state-api-report.html).

acknowledgments Writing this book has been one of the most fascinating journeys in my career, and I couldn’t have done it without the help and support of my family and an amazing team of colleagues. The book is dedicated to my wonderful wife, Jiwon, without whose constant encouragement and understanding I wouldn’t have been able to complete this book, and to our daughter, Ivy, who made sure I never had a dull moment in my schedule. I have benefited enormously from the people who contributed ideas for the book, helped me better understand the tools and protocols I use in it, and provided feedback on various chapters and drafts. Special thanks go to Dmitry Dygalo, Kelvin Meeks, Sebastián Ramírez Montaño, Chris Richardson, Jean Yang, Gajendra Deshpande, Oscar Islas, Mehdi Medjaoui, Ben Hutton, Andrej Baranovskij, Alex Mystridis, Roope Hakulinen, Steve Ardagh-Walter, Kathrin Björkelund, Thomas Dean, Marco Antonio Sanz, Vincent Vandenborne, and the amazing maintainers of Ariadne at Mirumee. Since 2020, I’ve presented drafts and ideas from the book at various conferences, including EuroPython, PyCon India, API World, API Specifications Conference, and various podcasts and meetups. I want to thank everyone who attended my presenta- tions and gave me valuable feedback. I also want to thank the attendants to my work- shops at microapis.io for their thoughtful comments on the book. I want to thank my acquisitions editor, Andy Waldron. Andy did a brilliant job helping me get my book proposal in good shape and keeping the book focused on rel- evant topics. He also supported me tirelessly to promote the book and helped me to reach a wider audience.xviii