Azure Cookbook Recipes to Create and Maintain Cloud Solutions in Azure (Reza Salehi)（Z-Library）

Reza Salehi Azure Cookbook Recipes to Create and Maintain Cloud Solutions in Azure

CLOUD COMPUTING “Reza Salehi’s knowledge of Microsoft Azure is extensive, and this book contains a tremendous amount of information about all aspects of Azure, endowing the reader with everything they need to succeed in developing Azure-based solutions.” —Pete Gallagher Avanade UK “This book is like a library of Azure services and how to use them. It provides broad, deep, and practical knowledge to get you started and keep you going into production and beyond.” —Barry Luijbregts a.k.a. Azure Barry Azure Cookbook Twitter: @oreillymedia linkedin.com/company/oreilly-media youtube.com/oreillymedia How do you deal with the challenges you face when using Azure? This practical guide provides more than 80 recipes to help you implement common Azure best practices in everyday scenarios. These recipes include key tasks such as managing Azure Storage Account and Cosmos DB permissions, creating smart applications with Cognitive Services, managing Azure RBAC, and much more. Author Reza Salehi has assembled real-world recipes that enable you to grasp Azure services and concepts quickly. Each recipe includes CLI scripts that you can execute in your own Azure account. These recipes also explain the approach and provide meaningful context. The solutions in this cookbook will take you beyond theory and help you understand Azure services in practice. You’ll find recipes that let you: • Store data in an Azure Storage Account or in a data lake • Work with relational and nonrelational databases in Azure • Manage role-based access control (RBAC) for Azure resources • Safeguard secrets in Azure Key Vault • Govern your Azure subscription using Azure Policy • Bring AI to your applications using Azure Cognitive Services Reza Salehi, an experienced consultant, has helped numerous organizations transition to the cloud and enhance the security, efficiency, and availability of their solutions. A 25-year veteran of the professional technology field, Reza has trained thousands of students in both in-person and online settings, including the O’Reilly learning platform. He’s been a Microsoft Certified Trainer (MCT) since 2008. US $79.99 CAN $99.99 ISBN: 978-1-098-13579-9

Reza Salehi Azure Cookbook Recipes to Create and Maintain Cloud Solutions in Azure Boston Farnham Sebastopol TokyoBeijing

978-1-098-13579-9 [LSI] Azure Cookbook by Reza Salehi Copyright © 2023 Reza Salehi. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com. Acquisitions Editor: Jennifer Pollock Development Editor: Jeff Bleiel Production Editor: Beth Kelly Copyeditor: Penelope Perkins Proofreader: Amnet Systems, LLC Indexer: Potomac Indexing, LLC Interior Designer: David Futato Cover Designer: Karen Montgomery Illustrator: Kate Dullea July 2023: First Edition Revision History for the First Edition 2023-06-22: First Release See http://oreilly.com/catalog/errata.csp?isbn=9781098135799 for release details. The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Azure Cookbook, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc. The views expressed in this work are those of the author and do not represent the publisher’s views. While the publisher and the author have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights.

Dedicated to my mother, Farideh, who gave me life and unconditional support.

Table of Contents Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix 1. Security. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1 Creating a New User in Your Azure Account 2 1.2 Creating a New Custom Role for Our User 5 1.3 Assigning Allowed Azure Resource Types in a Subscription 7 1.4 Assigning Allowed Locations for Azure Resources 10 1.5 Connecting to a Private Azure Virtual Machine Using Azure Bastion 11 1.6 Protecting Azure VM Disks Using Azure Disk Encryption 17 1.7 Blocking Anonymous Access to Azure Storage Blobs 19 1.8 Configuring an Azure Storage Account to Exclusively Use Azure AD Authorization 21 1.9 Storing and Retrieving Secrets from Azure Key Vault 23 1.10 Enabling Web Application Firewall (WAF) with Azure Application Gateway 25 2. Networking. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 2.1 Creating an Isolated Private Network by Provisioning an Azure Virtual Network 30 2.2 Creating a Network Layout in Azure Virtual Networks Using Subnets 32 2.3 Routing Network Traffic Using User-Defined Routes 34 2.4 Securing Azure Virtual Networks with Azure Firewall 36 2.5 Securing Azure Virtual Networks with Network Security Groups 40 2.6 Connecting Two Azure VNets Using Azure Network Peering 43 2.7 Verifying Azure VNet Connectivity Using Azure Network Watcher 46 v

3. Storage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 3.1 Using Azure Key Vault Keys to Configure Azure Storage Encryption at Rest 50 3.2 Controlling Azure Storage Network Access 53 3.3 Granting Limited Access to Azure Storage Using SAS Tokens 56 3.4 Granting Azure Function Apps Access to Azure Storage Using Managed Identity and RBAC 59 3.5 Creating and Storing Snapshots of Blob Objects 62 3.6 Creating and Accessing New File Versions 65 3.7 Using a Lifecycle Management Policy to Save Storage Account Costs 67 3.8 Using AzCopy to Upload Multiple Files to Azure Storage Blobs 71 3.9 Using AzCopy to Upload Multiple Files to Azure Storage File Shares 73 3.10 Protecting Azure Storage Blobs from Accidental Deletion 76 3.11 Protecting an Azure Storage Account from Deletion Using Azure Locks 78 4. Persisting Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 4.1 Creating a Cosmos DB NoSQL API Account 82 4.2 Creating a Cosmos DB Apache Gremlin (Graph) API Account 86 4.3 Configuring Azure Cosmos DB Firewall 88 4.4 Configuring Azure Cosmos DB Private Access 91 4.5 Granting Function Apps Access to Cosmos DB Using RBAC 94 4.6 Storing Tabular Data in Azure Storage Tables 98 4.7 Configuring Autoscale for an Azure Cosmos DB NoSQL API Container 100 4.8 Saving Costs on Multiple Azure SQL Single Databases with Varying and Unpredictable Usage Demands 103 4.9 Configuring Serverless Compute Tier for Azure SQL Single Databases 106 4.10 Configuring Azure SQL Firewall IP Rules 108 4.11 Configuring Azure SQL Firewall VNet Rules 111 4.12 Backing Up Azure SQL Single Databases into Azure Storage Blobs 113 5. Messaging and Events. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 5.1 Ingesting Streaming Data at Scale Using Event Hubs 118 5.2 Ingesting Telemetry Data from IoT Devices Using Azure IoT Hub 122 5.3 Implementing Communication Between Services Using Azure Storage Queues 125 5.4 Implementing Communication Between Services Using Azure Service Bus Queues 128 5.5 Implementing a Publish-Subscribe Pattern Using Azure Service Bus Topics 131 5.6 Queuing Newly Uploaded Blobs for Further Processing Using Azure Event Grid 133 vi | Table of Contents

6. Big Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 6.1 Processing Live Data Streams Using Azure Stream Analytics 140 6.2 Querying CSV File Content Using Azure Synapse Analytics Serverless Pool 147 6.3 Saving Costs on Idle Azure Synapse Dedicated SQL and Spark Pools 155 6.4 Processing Datafiles Using Azure Databricks 158 6.5 Performing ETL/ELT Operations on Big Data Using Azure Data Factory (ADF) 172 7. Azure Functions and Serverless Services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 7.1 Implementing a Web API Using Azure Functions 184 7.2 Invoking an Azure Function on a Schedule Using the Timer Trigger 190 7.3 Invoking an Azure Function on Blob Upload Using the Blob Trigger 195 7.4 Inserting Function App Output into Azure Cosmos DB 201 7.5 Establishing Communication Between Azure Functions Using the Service Bus Queue Trigger 206 7.6 Restricting Network Access to Azure Function Apps 213 8. Azure App Service. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217 8.1 Deploying a Web Application to Azure App Services Using ZipDeploy 218 8.2 Deploying a Web Application to Azure App Service Using FTP 221 8.3 Deploying a Web Application from a Public GitHub Repository to Azure App Service 224 8.4 Configuring Autoscaling for Your Azure App Service Plan 226 8.5 Restricting Network Access to an Azure App Service 230 8.6 Hosting Static Websites in Azure Storage 235 9. Containers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239 9.1 Building and Pushing a Container Image to Azure Container Registry 241 9.2 Pulling and Running a Docker Image in Azure Container Instances 246 9.3 Pulling and Running a Containerized Website in Azure App Services 248 9.4 Pulling and Running a Containerized Website in Azure Container Apps 251 9.5 Limiting Azure Container Registry Public Network Access 254 10. Azure Cognitive Services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261 10.1 Provisioning a Multi-Service Azure Cognitive Services Resource 263 10.2 Converting Text to Speech 266 10.3 Translating Text 268 10.4 Performing Sentiment Analysis on Text 271 10.5 Detecting Objects in Images 274 10.6 Creating Captions for Images 277 Table of Contents | vii

10.7 Extracting Text from Images Using OCR 279 10.8 Detecting PII in Text 282 11. Management and Monitoring. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285 11.1 Managing Azure Resource Tags 286 11.2 Estimating Costs for Azure Resources 289 11.3 Monitoring Costs of Provisioned Azure Resources 295 11.4 Collecting Platform Logs for an Azure Resource 297 11.5 Analyzing Azure Monitor Platform Logs 302 Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309 viii | Table of Contents

Preface As an IT professional, I have always been fascinated by the world of cloud computing and the endless possibilities it presents. Azure, in particular, stood out to me as a plat‐ form that offered a comprehensive range of services and tools for businesses of all sizes. With its ability to simplify the development, deployment, management, and scaling of applications, it has a significant potential to revolutionize the way organiza‐ tions operate. Global end-user spending on public cloud services is expected to reach $591.8 billion in 2023, up from $490.3 billion in 2022, a 20.7% growth. This creates a huge demand for cloud professionals to fill in the job market talent gap. I had the opportunity to work on a wide range of Azure solutions for businesses of all sizes in the past few years, giving me a perspective on challenges that companies can face in a cloud jour‐ ney. As a Microsoft Certified Trainer (MCT), I had the privilege of sharing this per‐ spective with thousands of IT professionals. This book aims to enable you to quickly acquire hands-on expertise with the key Azure services and concepts. I hope you use this book as a reference to solve common, yet fundamental, challenges in your Azure cloud projects. Who Should Read This Book This book is for cloud architects, developers, and engineers at all experience levels. Beginners will use the recipes in this book to familiarize themselves with the funda‐ mental Azure services and gain hands-on experience with them. Expert cloud profes‐ sionals will use this book to enhance their Azure projects, solve common architecture problems, and gain further perspectives. The recipes in this book showcase how mul‐ tiple Azure services can be combined to deliver immediate value for you, regardless of your Azure expertise level. This book is for you, if you like to learn about a diverse set of Azure services for security, networking, big data, storage, databases, AI, con‐ tainers, serverless, and web apps. ix

Why I Wrote This Book Writing this book was my way of contributing to the cloud revolution by sharing my experience with other professionals and enthusiasts. I wanted to create a resource that would not only provide an overview of a wide variety of Azure platform services but also guide readers through practical, hands-on scenarios and use cases. My aim was to equip professionals with the necessary skills to leverage Azure’s capabilities and help organizations drive their digital transformation. By writing this book, I hope to inspire others to fully embrace the power of Azure and the future of cloud computing. Navigating This Book The hands-on recipes in this book are organized into the following chapters: • Chapter 1, “Security”, provides methods to improve the security of Azure resources through RBAC, role-based access control, and network firewalls. • Chapter 2, “Networking”, reviews Azure Virtual Network (VNet) security, rout‐ ing, and monitoring. • Chapter 3, “Storage”, provides recipes for Azure storage accounts, enabling you to optimize cost, secure your data, and protect it against accidental deletion. • Chapter 4, “Persisting Data”, provides guidelines to configure and protect the main Azure relational and NoSQL databases, Azure SQL and Azure Cosmos DB. • Chapter 5, “Messaging and Events”, enables you to set up reliable messaging between your services and solutions using Azure’s messaging suite of services. • Chapter 6, “Big Data”, introduces Azure services, including Azure Stream Analyt‐ ics, Azure Synapse Analytics, Azure Databricks, and Azure Data Factory, designed to mine insights from your big data. • Chapter 7, “Azure Functions and Serverless Services”, provides recipes to imple‐ ment microservices using Azure Function Apps. • Chapter 8, “Azure App Service”, provides recipes to configure autoscaling, secure network access, and deploy App Services using several methods. • Chapter 9, “Containers”, introduces Azure services designed to host and run con‐ tainerized applications in Azure. • Chapter 10, “Azure Cognitive Services”, helps you develop smart applications by using Azure Cognitive Services. Recipes in this chapter enable you to gain insights from images, audio, and text content using AI-backed services. • Chapter 11, “Management and Monitoring”, introduces tools to monitor and control Azure service costs and then reviews Azure Monitor platform logs. x | Preface

What You Will Need Here are the resources and tools you need to get the most out of this book: • Azure account (subscription): — See the Azure documentation for setup instructions. You’ll need a user with Owner (administrative) access over your subscription. • Personal computer with Windows, macOS, or Linux OS • Software: — A supported web browser (e.g., Microsoft Edge or Google Chrome). — Terminal with Bash or Z shell (Zsh). — Git (see the installation instructions). — A code editor (e.g., VS Code, VSCodium). You can access the bash terminal from VS Code. — Azure CLI version 2.42 or later (see the installation instructions). • The book repository: — Clone it to a local folder on your machine. Getting Started The recipes in this book use Azure CLI to provision and manage Azure resources. This section provides details to help you run Azure CLI commands on your worksta‐ tion. Feel free to skip this section if you are already comfortable with Azure CLI. Although many Microsoft Azure services offer free tiers, Azure is a paid cloud service and you will be charged for any services that you create in the paid pricing tiers. To avoid getting charged, clean up (delete) any provisioned resource after completing a recipe. See the recipe cleanup section in the book repository for details. Azure Account Setup You need a user with the subscription administrator (Owner) permissions. This user account enables you to create resources and identities as well as configure the permis‐ sions that are needed to complete the recipes in this book. Your Azure subscription comes with a default administrator account, which can be used. You can also create a Co-Administrator user if preferred. Preface | xi

General Workstation Setup Instructions 1. The CLI scripts used in this book can be found in the book repository. Create a folder called AzureCookbook in any directory you prefer and set it as the current folder: mkdir ~/AzureCookbook cd ~/AzureCookbook 2. Now, let’s clone the book repository into your current folder: git clone https://github.com/zaalion/AzureCookbook.git 3. Microsoft Azure provides several regions (locations). We use “eastus” in this book, but you can choose any region you like. Run the following command to get a list of all Azure regions: az account list-locations --output table 4. Store your desired region name in the following variable: region="eastus" 5. Set a default location for Azure CLI commands. This location will be used if no location is passed to the CLI command: az config set defaults.location=eastus 6. Log in to Azure CLI using your Azure subscription administrator username and password. Replace <username> and <password> with your admin account credentials: az login -u "<username>" -p "<password>" Running az login without passing account credentials will open a browser window where you can enter your username/password. 7. You should create your Azure resources in a parent container called a resource group. Before starting each recipe, you have to create a new resource group. Replace <resource-group-name> with the desired name. Store the resource group name in the rgName variable as shown in the following command. You will use rgName in every recipe: xii | Preface

rgName="<resource-group-name>" az group create \ --location $region \ --name $rgName You can alternatively run your commands in the Azure Cloud Shell, as shown in Figure P-1. Figure P-1. Azure Cloud Shell Conventions Used in This Book The following typographical conventions are used in this book: Italic Indicates new terms, URLs, email addresses, filenames, and file extensions. Constant width Used for program listings, as well as within paragraphs to refer to program ele‐ ments such as variable or function names, databases, data types, environment variables, statements, and keywords. Constant width italic Shows text that should be replaced with user-supplied values or by values deter‐ mined by context. Preface | xiii

This element signifies a tip or suggestion. This element signifies a general note. This element indicates a warning or caution. Using Code Examples Supplemental material (code examples, exercises, etc.) is available for download at https://github.com/zaalion/AzureCookbook. If you have a technical question or a problem using the code examples, please send email to support@oreilly.com. This book is here to help you get your job done. In general, if example code is offered with this book, you may use it in your programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require permission. We appreciate, but generally do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: “Azure Cookbook by Reza Salehi (O’Reilly). Copyright 2023 Reza Salehi, 978-1-098-13579-9.” If you feel your use of code examples falls outside fair use or the permission given above, feel free to contact us at permissions@oreilly.com. xiv | Preface

O’Reilly Online Learning For more than 40 years, O’Reilly Media has provided technol‐ ogy and business training, knowledge, and insight to help companies succeed. Our unique network of experts and innovators share their knowledge and expertise through books, articles, and our online learning platform. O’Reilly’s online learning platform gives you on-demand access to live training courses, in-depth learning paths, interactive coding environments, and a vast collection of text and video from O’Reilly and 200+ other publishers. For more information, visit https://oreilly.com. How to Contact Us Please address comments and questions concerning this book to the publisher: O’Reilly Media, Inc. 1005 Gravenstein Highway North Sebastopol, CA 95472 800-889-8969 (in the United States or Canada) 707-829-7019 (international or local) 707-829-0104 (fax) support@oreilly.com https://www.oreilly.com/about/contact.html We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at https://oreil.ly/azure-cookbook. For news and information about our books and courses, visit https://oreilly.com. Find us on LinkedIn: https://linkedin.com/company/oreilly-media Follow us on Twitter: https://twitter.com/oreillymedia Watch us on YouTube: https://youtube.com/oreillymedia Acknowledgments Thank you to Jennifer Pollock for giving me the opportunity to showcase my techni‐ cal writing skills and bring my book to life. I am grateful for your support. I appreciate the O’Reilly Media team. Thank you to the development editor, Jeff Bleiel. Your guidance, suggestions, and insights enabled me to refine my work and bring this book to life. Thank you also to the production editor, Elizabeth Kelly, for getting this book in the best shape for release. Preface | xv

Thank you to the book reviewers for taking the time to share your thoughts and opinions on my work: Dipal Choski, George Mount, Hakan Silfvernagel, and Peter De Tender. Your feedback is greatly appreciated. xvi | Preface

CHAPTER 1 Security Data breaches happen frequently these days. Hacker and malicious users target IT systems in small, mid-sized, and big organizations. These attacks cost millions of dol‐ lars every year, but cost is not the only damage. Targeted companies will be on the news for days, weeks, or even years, and they may suffer permanent damage to their reputation and customer base. In most cases, lawsuits will follow. You might be under the impression that public cloud services are very secure. After all, companies such as Microsoft spend millions of dollars improving their platform security. But applications and data systems hosted in Microsoft’s public cloud (as well as clouds of other providers) are not immune to cyberattacks. In fact, they are more prone to data breaches because of the public nature of the cloud. It is critical to understand that cloud security is a common responsibility shared between Microsoft Azure and you. Azure provides data center physical security, guidelines, documentation, and powerful tools and services to help you protect your workloads. It is your responsibility to correctly configure resource security. For exam‐ ple, Azure Cosmos DB can be configured to accept traffic from only a specific net‐ work, but the default behavior allows all clients, even from the public internet. Cloud security is an evolving subject. Microsoft Azure continues to introduce new security features to protect your workloads against new threats. Microsoft maintains the Azure security best practices and patterns documentation, which you can consult for the most up-to-date security best practices. We chose security as the topic for the first chapter of this book to deliver this impor‐ tant message: Security must come first! You should have security in mind while designing, implementing, and supporting your cloud projects. In this chapter, we’ll 1

share useful recipes showing how to secure key Azure services and then use these recipes’ outcomes throughout the book. We cover important Azure security topics in this chapter. However, it is not possible to cover all topics related to Azure security. Azure services and capabilities continue to evolve on a daily basis. You must consult the Microsoft documentation for a complete and updated list. Workstation Configuration You will need to prepare your workstation before starting on the recipes in this chap‐ ter. Follow “What You Will Need” on page xi to set up your machine to run Azure CLI commands. Clone the book’s GitHub repository using the following command: git clone https://github.com/zaalion/AzureCookbook.git 1.1 Creating a New User in Your Azure Account Problem The Owner (administrator) account has far more permissions than are needed for everyday development tasks. You need to create a new user account for a developer with just enough permissions to complete assigned tasks. Solution First, create a new user in your Azure Active Directory (Azure AD). Then assign the Contributor role-based access control (RBAC) role to that user, so enough permis‐ sions are assigned without granting this user the same permission level as the Owner. An architecture solution diagram is shown in Figure 1-1. Figure 1-1. Assigning a built-in role to Azure AD users/groups 2 | Chapter 1: Security