AWS Cookbook Recipes for Success on AWS (John Culkin and Mike Zazon) (z-library.sk, 1lib.sk, z-lib.sk)

C ulkin & Za zon A W S C ookb ook A W S C ookb ook AWS Cookbook Recipes for Success on AWS John Culkin & Mike Zazon Foreword by Jeff Barr

CLOUD “There’s something sorely missing from official AWS documentation: a sense of reality. Most of us aren’t attempting to win points for collecting as many AWS services as we can; we’re trying to complete a task somewhere that isn’t a whiteboard. AWS Cookbook speaks to real users with a collection of recipes and pragmatic examples we can all benefit from.” —Corey Quinn Chief Cloud Economist, The Duckbill Group AWS Cookbook ISBN: 978-1-492-09260-5 US $69.99 CAN $92.99 This practical guide provides over 70 self-contained recipes to help you creatively solve common AWS challenges you’ll encounter on your cloud journey. If you’re comfortable with rudimentary scripting and general cloud concepts, this cookbook provides what you need to address foundational tasks and create high-level capabilities. Authors John Culkin and Mike Zazon share real-world examples that incorporate best practices. Each recipe includes a diagram to visualize the components. Code is provided so that you can safely execute in an AWS account to ensure solutions work as described. From there, you can customize the code to help construct an application or fix an existing problem. Each recipe also includes a discussion to provide context, explain the approach, and challenge you to explore the possibilities further. Go beyond theory and learn the details you need to successfully build on AWS. The recipes help you: • Redact personal identifiable information (PII) from text using Amazon Comprehend • Automate password rotation for Amazon RDS databases • Use VPC Reachability Analyzer to verify and troubleshoot network paths • Lock down Amazon Simple Storage Service (S3) buckets • Analyze AWS Identity and Access Management policies • Autoscale a containerized service John Culkin is a senior solutions architect at AWS. He now focuses on creating transformative business solutions that utilize cloud services. Mike Zazon is a senior cloud architect at AWS. His passion for technology education blossomed while working in an engineering research university setting. Twitter: @oreillymedia linkedin.com/company/oreilly-media youtube.com/oreillymedia C ulkin & Za zon

Praise for AWS Cookbook There’s something sorely missing from official AWS documentation: a sense of reality. Most of us aren’t attempting to win points for collecting as many AWS services as we can; we’re trying to complete a task somewhere that isn’t a whiteboard. AWS Cookbook speaks to real users with a collection of recipes and pragmatic examples we can all benefit from. —Corey Quinn, Chief Cloud Economist, The Duckbill Group Inside you’ll find a great deal of information on typical AWS use cases plus a reference implementation that’s easy to follow. If you like to learn AWS concepts in a practice- driven, example-based, hands-on manner, I highly recommend this book. —Gaurav Raje, author of Security and Microservice Architecture on AWS I’ve never read a book packed so densely with ninja level tips and tricks for AWS; it’s the book I wish I had five years ago. If you use AWS day to day, you need this in your toolkit, not only for the things it contains but also for the inspiration it provides. In my view, it’s the best AWS book there is. —Adrian Cantrill, AWS Trainer, learn.cantrill.io Putting AWS into practice with hands-on experience is the difference between cloud literacy and cloud fluency. AWS Cookbook serves up practical scenarios for working in the cloud to help individuals level up their career. —Drew Firment, AWS Community Hero and Head Enterprise Strategist, Pluralsight

John Culkin and Mike Zazon AWS Cookbook Recipes for Success on AWS

978-1-492-09260-5 [LSI] AWS Cookbook by John Culkin and Mike Zazon Copyright © 2022 Culkins Coffee Shop LLC and Mike Zazon. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com. Acquisitions Editor: Jennifer Pollock Development Editor: Virginia Wilson Production Editor: Christopher Faucher Copyeditor: nSight, Inc. Proofreader: Sharon Wilkey Indexer: Ellen Troutman-Zaig Interior Designer: David Futato Cover Designer: Karen Montgomery Illustrator: Kate Dullea December 2021: First Edition Revision History for the First Edition 2021-12-02: First Release See http://oreilly.com/catalog/errata.csp?isbn=9781492092605 for release details. The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. AWS Cookbook, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc. The views expressed in this work are those of the authors and do not represent the publisher’s views. While the publisher and the authors have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the authors disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights.

Dedicated to my father, who taught me that a spreadsheet could be used for much more than totaling up columns. —John Dedicated to my aunt, Judy Dunn. Thank you for the Tandy 1000 PC that sparked my fascination with computer programming and technology. —Mike

Table of Contents Foreword. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii 1. Security. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.0 Introduction 1 1.1 Creating and Assuming an IAM Role for Developer Access 2 1.2 Generating a Least Privilege IAM Policy Based on Access Patterns 6 1.3 Enforcing IAM User Password Policies in Your AWS Account 9 1.4 Testing IAM Policies with the IAM Policy Simulator 13 1.5 Delegating IAM Administrative Capabilities Using Permissions Boundaries 17 1.6 Connecting to EC2 Instances Using AWS SSM Session Manager 25 1.7 Encrypting EBS Volumes Using KMS Keys 30 1.8 Storing, Encrypting, and Accessing Passwords Using Secrets Manager 33 1.9 Blocking Public Access for an S3 Bucket 36 1.10 Serving Web Content Securely from S3 with CloudFront 39 2. Networking. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 2.0 Introduction 43 2.1 Defining Your Private Virtual Network in the Cloud by Creating an Amazon VPC 44 2.2 Creating a Network Tier with Subnets and a Route Table in a VPC 47 2.3 Connecting Your VPC to the Internet Using an Internet Gateway 51 2.4 Using a NAT Gateway for Outbound Internet Access from Private Subnets 55 2.5 Granting Dynamic Access by Referencing Security Groups 59 vii

2.6 Using VPC Reachability Analyzer to Verify and Troubleshoot Network Paths 63 2.7 Redirecting HTTP Traffic to HTTPS with an Application Load Balancer 67 2.8 Simplifying Management of CIDRs in Security Groups with Prefix Lists 74 2.9 Controlling Network Access to S3 from Your VPC Using VPC Endpoints 78 2.10 Enabling Transitive Cross-VPC Connections Using Transit Gateway 82 2.11 Peering Two VPCs Together for Inter-VPC Network Communication 88 3. Storage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 3.0 Introduction 93 3.1 Using S3 Lifecycle Policies to Reduce Storage Costs 94 3.2 Using S3 Intelligent-Tiering Archive Policies to Automatically Archive S3 Objects 97 3.3 Replicating S3 Buckets to Meet Recovery Point Objectives 100 3.4 Observing S3 Storage and Access Metrics Using Storage Lens 105 3.5 Configuring Application-Specific Access to S3 Buckets with S3 Access Points 110 3.6 Using Amazon S3 Bucket Keys with KMS to Encrypt Objects 114 3.7 Creating and Restoring EC2 Backups to Another Region Using AWS Backup 117 3.8 Restoring a File from an EBS Snapshot 125 3.9 Replicating Data Between EFS and S3 with DataSync 128 4. Databases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 4.0 Introduction 133 4.1 Creating an Amazon Aurora Serverless PostgreSQL Database 134 4.2 Using IAM Authentication with an RDS Database 140 4.3 Leveraging RDS Proxy for Database Connections from Lambda 146 4.4 Encrypting the Storage of an Existing Amazon RDS for MySQL Database 153 4.5 Automating Password Rotation for RDS Databases 157 4.6 Autoscaling DynamoDB Table Provisioned Capacity 163 4.7 Migrating Databases to Amazon RDS Using AWS DMS 167 4.8 Enabling REST Access to Aurora Serverless Using RDS Data API 171 5. Serverless. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177 5.0 Introduction 177 5.1 Configuring an ALB to Invoke a Lambda Function 179 5.2 Packaging Libraries with Lambda Layers 181 5.3 Invoking Lambda Functions on a Schedule 185 5.4 Configuring a Lambda Function to Access an EFS File System 188 5.5 Running Trusted Code in Lambda Using AWS Signer 191 viii | Table of Contents

5.6 Packaging Lambda Code in a Container Image 194 5.7 Automating CSV Import into DynamoDB from S3 with Lambda 198 5.8 Reducing Lambda Startup Times with Provisioned Concurrency 201 5.9 Accessing VPC Resources with Lambda 204 6. Containers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207 6.0 Introduction 207 6.1 Building, Tagging, and Pushing a Container Image to Amazon ECR 209 6.2 Scanning Images for Security Vulnerabilities on Push to Amazon ECR 214 6.3 Deploying a Container Using Amazon Lightsail 217 6.4 Deploying Containers Using AWS Copilot 220 6.5 Updating Containers with Blue/Green Deployments 223 6.6 Autoscaling Container Workloads on Amazon ECS 227 6.7 Launching a Fargate Container Task in Response to an Event 231 6.8 Capturing Logs from Containers Running on Amazon ECS 235 7. Big Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241 7.0 Introduction 241 7.1 Using a Kinesis Stream for Ingestion of Streaming Data 242 7.2 Streaming Data to Amazon S3 Using Amazon Kinesis Data Firehose 244 7.3 Automatically Discovering Metadata with AWS Glue Crawlers 249 7.4 Querying Files on S3 Using Amazon Athena 256 7.5 Transforming Data with AWS Glue DataBrew 261 8. AI/ML. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267 8.0 Introduction 267 8.1 Transcribing a Podcast 268 8.2 Converting Text to Speech 270 8.3 Computer Vision Analysis of Form Data 272 8.4 Redacting PII from Text Using Comprehend 275 8.5 Detecting Text in a Video 278 8.6 Physician Dictation Analysis Using Amazon Transcribe Medical and Comprehend Medical 281 8.7 Determining Location of Text in an Image 284 9. Account Management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287 9.0 Introduction 287 9.1 Using EC2 Global View for Account Resource Analysis 288 9.2 Modifying Tags for Many Resources at One Time with Tag Editor 290 9.3 Enabling CloudTrail Logging for Your AWS Account 295 9.4 Setting Up Email Alerts for Root Login 298 Table of Contents | ix

9.5 Setting Up Multi-Factor Authentication for a Root User 300 9.6 Setting Up AWS Organizations and AWS Single Sign-On 305 Appendix. Fast Fixes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311 Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315 x | Table of Contents

Foreword As part of the Amazon Web Services (AWS) team since the beginning, I have been able to watch it grow in scale, richness, and complexity from a unique vantage point. Even after writing thousands of blog posts and millions of words, I learn something new and useful about AWS just about every day. With well over two hundred services in production and more launching regularly, AWS could easily leave you feeling overwhelmed. In addition to tens of thousands of pages of official AWS documentation, bloggers, AWS Heroes, AWS Partners, and oth‐ ers have created innumerable pieces of content—including blog posts, videos, webi‐ nars, overviews, and code samples. While there’s no substitute for having a full and complete understanding of a particu‐ lar AWS service, the reality is that you often simply need to solve a “point” problem. Even after you understand a service, remembering how to use it to solve that problem can be a challenge—at least it is for me. And that is where this cookbook comes in. Because of its broad selection of topics and carefully chosen recipes, I am confident that you will be able to quickly find one that addresses your immediate need and to put it into practice in short order. You can solve your problem, refresh your knowledge of that aspect of AWS, and move for‐ ward to create value for your customers! My favorite aspect of this book is that it does not hand-wave past any of the details. Each recipe assumes that you start fresh and then helps you to cook up a perfectly seasoned solution. Nothing is left to chance, and you can use the recipes as is in most cases. The recipes also cover the all-important cleanup phase and ensure that you leave your AWS environment as you found it. xi

Where appropriate, the recipes use the AWS Cloud Development Kit (CDK) and include all of the necessary “moving parts.” The CDK provides a double benefit; in addition to helping you to move forward more quickly, these CDK elements can help you learn more about how to put infrastructure as code (IaC) into practice. Most cookbooks are designed to be browsed and savored, and this one is no excep‐ tion. Flip through it, read an entire chapter, or use just a recipe or two, as you wish. I also recommend that you go through all of Chapter 1, just to make sure that your environment is set up and ready to go. Then, when you are presented with a problem to solve, find the appropriate recipe, put it into practice, and reap the benefits. —Jeff Barr —VP and Chief Evangelist at AWS Seattle, WA November 2021 xii | Foreword

Preface The vast majority of workloads will go to the cloud. We’re just at the beginning—there’s so much more to happen. —Andy Jassy Cloud usage has been gaining traction with enterprises and small businesses over the last decade and continues to accelerate. Gartner said the worldwide infrastructure as a service (IaaS) public cloud services market grew 40.7% in 2020. The rapid growth of the cloud has led to a huge demand for cloud skills by many organizations. Many IT professionals understand the basic concepts of the cloud but want to become more comfortable working in the cloud. This gap between the supply and demand of cloud skills presents a significant opportunity for individuals to level up their career. Through our combined 20+ years of cloud experience, we have had the benefit of working on Amazon Web Services (AWS) projects in many different roles. We have provided guidance to hundreds of developers on how and when to use AWS services. This has allowed us to understand the common challenges and easy wins of the cloud. We would like to share these lessons with you and give you a leg up for your own advancement. We wrote this book to share some of our knowledge and enable you to quickly acquire useful skills for working in the cloud. We hope that you will find yourself using this book as reference material for many years to come. Who This Book Is For This book is for developers, engineers, and architects of all levels, from beginner to expert. Beginners will learn cloud concepts and become comfortable working with cloud services. Experts will be able to examine code used to stand up recipe founda‐ tions, explore new services, and gain additional perspectives. If the plethora of cloud services and combinations seem overwhelming to you, then this book is for you. The recipes in this book aim to provide “Hello, World” proofs of concept and components of enterprise-grade applications. This will be accomplished using common use cases with guided walk-throughs of scenarios that you can directly apply to your current or xiii

future work. These curated and experience-building recipes are meant to demystify services and will immediately deliver value, regardless of your AWS experience level. What You Will Learn In addition to opening up new career opportunities, being able to harness the power of AWS will give you the ability to create powerful systems and applications that solve many interesting and demanding problems in our world today. Would you like to handle 60,000 cyber threats per second using AWS machine learning like Siemens does? Or reduce your organization’s on-premises footprint and expand its use of microservices like Capital One has? If so, the practical examples in this book will help expedite your learning by providing tangible examples showing how you can put the building blocks of AWS together to form practical solutions that address common scenarios. The on-demand consumption model, vast capacity, advanced capabilities, and global footprint of the cloud create new possibilities that need to be explored. The Recipes We break the book into chapters that focus on general areas of technology (e.g., secu‐ rity, networking, artificial intelligence, etc.). The recipes contained within the chap‐ ters are bite-sized, self-contained, and easily consumable. Recipes vary in length and complexity. Each recipe has a problem statement, solution (with diagram), and dis‐ cussion. Problem statements are tightly defined to avoid confusion. Solutions contain required preparation and steps to walk you through the work needed to accomplish the goal. When appropriate, explicit validation checks will be provided. We’ve also added extra challenges to the recipes to help you advance your learning if you wish to do so. Finally, we end each recipe with a short discussion to help you understand the solution and why it matters, suggestions to extend the solution, and ways to utilize it for real impact. To keep your AWS bill low and keep your account tidy, each recipe has cleanup steps provided in the repositories associated with the book. Each chapter has its own repository at https://github.com/awscookbook. The reposi‐ tory contains preparation steps for easy copying and pasting, required files, and infra‐ structure as code. We have also created GitHub templates for reporting bugs and sug‐ gesting new recipes. We encourage you to leverage GitHub to submit issues, create requests for new recipes, and submit your own pull requests. We will actively maintain the chapter repositories with updates for recipe steps and code in the README files of each recipe. Be sure to check these for any new or alternative xiv | Preface

approaches. We look forward to interacting with you on GitHub with new fun chal‐ lenges and hints to assist you. Some recipes are “built from scratch,” and others include preparation steps to allow you to interact with common scenarios seen in the real world. We have provided code to enable you to easily deploy the prerequisites. For example, Recipe 6.5, “Updating Containers with Blue/Green Deployments”, assumes that you are a container devel‐ oper creating an application deployment that requires an existing network stack. When prerequisites exist, they can be “pre-baked” with preparation steps using code provided in the repositories. When substantial preparation for a recipe is needed, you will use the AWS Cloud Development Kit (CDK), which is a fantastic tool for intelli‐ gently defining and declaring infrastructure. The majority of the recipes are CLI based; when appropriate, we use console walk-throughs including screenshots or descriptive text. There are many ways to achieve similar outcomes on AWS; this book will not be an exhaustive list. Many factors will dictate the best overall solution for your use case. We have selected recipe top‐ ics to help you learn about AWS and make the best choices for your specific needs. You’ll find recipes for things like the following: • Redacting personally identifiable information (PII) from text by using Amazon Comprehend • Automating password rotation for Amazon Relational Database Service (RDS) databases • Using VPC Reachability Analyzer to verify and troubleshoot network paths Along with the recipes, we also provide short lines of code in the Appendix that will quickly accomplish valuable and routine tasks. We feel that these are great tidbits to add to your cloud toolbox. AWS has a free tier, but implementing recipes in this book could incur costs. We provide cleanup instructions, but you are responsi‐ ble for any costs in your account. We recommend checking out the Well-Architected Labs developed by AWS on expenditure aware‐ ness and leveraging AWS Budgets actions to control costs. Preface | xv

What You Will Need Here are the requirements to get started and some tips on where to find assistance: • AWS account — Setup instructions — An IAM user with console and programmatic access — Administrator privileges for your IAM user • Personal computer/laptop • Software — Web browser (e.g., Microsoft Edge, Google Chrome, or Mozilla Firefox) — Terminal with bash or Z shell (Zsh) — Git — Install instructions — Homebrew (optional but recommended to install other requirements) — Install instructions — Code editor (e.g., VSCodium or AWS Cloud9) — Recommended install: brew install --cask vscodium — AWS CLI version 2 (2.1.26 or later) — Install guide — Recommended install: brew install awscli@2 — Python 3.7.9 (and pip) or later — Example install: brew install python@3.7 — AWS Cloud Development Kit version 2.0 or later — Getting started guide — Recommended install: brew install npm and npm i -g aws-cdk@next • Recommended: Create a folder in your home directory called AWSCookbook. This will allow you to clone each chapter’s repository in one place: AWSCookbook:$ tree -L 1 . ├── AccountManagement ├── ArtificialIntelligence ├── BigData xvi | Preface

At the time of publishing, the AWS CDK has two versions: version 1 and version 2 (developer preview). The code we have provided is written for version 2. You can find out more information about how to migrate to and install CDK version 2 in this AWS CDK v2 article. Getting Started This section provides examples of techniques and approaches we perform throughout the book to make the recipe steps easier to follow. You can skip over these topics if you feel comfortable with them. You can always come back and reference this section. Setups In addition to the installation of the prerequisites listed previously, you will need the following access. AWS account setup You will need a user with administrative permissions. Some of the recipes require the ability to create AWS Identity and Access Management (IAM) resources. You can fol‐ low the AWS guide for creating your first IAM admin user and user group. General workstation setup steps for CLI recipes We have created a group of code repositories available at https://github.com/awscook book. Create a folder called AWSCookbook in your home directory (or any place of your choosing) and cd there: mkdir ~/AWSCookbook && cd ~/AWSCookbook This will give you a place to check out chapter repositories (e.g., Security): git clone https://github.com/AWSCookbook/Security Set and export your default Region in your terminal: export AWS_REGION=us-east-1 AWS offers many Regions across the world for cloud deployments. We’ll be using the us-east-1 Region for simplicity. As long as the services are available, there is no reason these recipes won’t work in other Regions. AWS has a list of Regions and services. Set your AWS ACCOUNT_ID by parsing output from the aws sts get-caller- identity operation: Preface | xvii

AWS_ACCOUNT_ID=$(aws sts get-caller-identity \ --query Account --output text) The aws sts get-caller-identity operation “returns details about the IAM user or role whose credentials are used to call the operation.” Validate AWS Command Line Interface (AWS CLI) setup and access: aws ec2 describe-instances If you don’t have any EC2 instances deployed, you should see output similar to the following: { "Reservations": [] } AWS CLI version 2 will by default send command output with multiple lines to less in your terminal. You can type q to exit. If you want to override this behavior, you can modify your ~/.aws/ config file to remove this default functionality. AWS CloudShell is a browser-based terminal that you can use to quickly create a terminal environment in your authenticated AWS Console session to run AWS CLI commands from. By default, it uses the identity of your browser session to interact with the AWS APIs. Many of the recipes can be run using CloudShell. You can use CloudShell to run recipe steps, clean up commands, and other AWS CLI commands as your authenticated user, if you do not want to create a session that you use in your own local terminal environ‐ ment on your workstation. Techniques and Approaches Used in This Book The next few sections will explain and give examples of some ways of using the CLI to help you with recipes. Querying outputs, environment variables, and command substitution Sometimes when subsequent commands depend on outputs from the command you are currently running. The AWS CLI provides the ability for client-side filtering of output. At times, we will set environment variables that contain these outputs by lev‐ eraging command substitution. xviii | Preface