Page
1
Monitoring Cloud-Native Applications Lead Agile Operations Confidently Using Open Source Software — Mainak Chakraborty Ajit Pratap Kundan
Page
2
Monitoring Cloud-Native Applications Lead Agile Operations Confidently Using Open Source Software Mainak Chakraborty Ajit Pratap Kundan
Page
3
Monitoring Cloud-Native Applications ISBN-13 (pbk): 978-1-4842-6887-2 ISBN-13 (electronic): 978-1-4842-6888-9 https://doi.org/10.1007/978-1-4842-6888-9 Copyright © 2021 by Mainak Chakraborty and Ajit Pratap Kundan This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Trademarked names, logos, and images may appear in this book. Rather than use a trademark symbol with every occurrence of a trademarked name, logo, or image we use the names, logos, and images only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein. Managing Director, Apress Media LLC: Welmoed Spahr Acquisitions Editor: Aditee Mirashi Development Editor: Matthew Moodie Coordinating Editor: Aditee Mirashi Cover designed by eStudioCalamar Cover image designed by Freepik (www.freepik.com) Distributed to the book trade worldwide by Springer Science+Business Media New York, 1 New York Plaza, Suite 4600, New York, NY 10004-1562, USA. Phone 1-800-SPRINGER, fax (201) 348-4505, e-mail orders-ny@ springer-sbm.com, or visit www.springeronline.com. Apress Media, LLC is a California LLC and the sole member (owner) is Springer Science + Business Media Finance Inc (SSBM Finance Inc). SSBM Finance Inc is a Delaware corporation. For information on translations, please e-mail booktranslations@springernature.com; for reprint, paperback, or audio rights, please e-mail bookpermissions@springernature.com. Apress titles may be purchased in bulk for academic, corporate, or promotional use. eBook versions and licenses are also available for most titles. For more information, reference our Print and eBook Bulk Sales web page at http://www.apress.com/bulk-sales. Any source code or other supplementary material referenced by the author in this book is available to readers on GitHub via the book’s product page, located at www.apress.com/978-1-4842-6887-2. For more detailed information, please visit http://www.apress.com/source- code. Printed on acid-free paper Mainak Chakraborty Gurugram, India Ajit Pratap Kundan Faridabad, India
Page
4
To Ritu, my best friend, for believing in me. To Jay Gopal and Gita Chakraborty, my parents, for their unwavering support. —Mainak Chakraborty
Page
5
(This page has no text content)
Page
6
(This page has no text content)
Page
7
(This page has no text content)
Page
8
ix About the Authors Mainak Chakraborty is a senior solutions architect at a leading public cloud company, specializing in cloud management and automation tools. He has been instrumental in shaping the cloud journey of customers across industry segments whether they be established enterprises or born-in-the-cloud startups. Mainak is an open source enthusiast and regularly presents at industry technical events on his favorite topics of automation, cloud native applications and cloud computing. Ajit Pratap Kundan stands at the leading edge of the innovative technologies of todays’ information technology world. He’s worked with companies like HPE, VMware, Novell and helped customers in transforming their datacenters through software-defined services. Ajit is a valued author on cloud technologies and has authored two books—VMware Cross-Cloud Architecture and Intelligent Automation with VMware published by Packt—and has reviewed one book, Deep Learning with Pytorch.
Page
9
xi About the Technical Reviewer Amit Agrawal is a principal data scientist, architect and researcher delivering solutions in the fields of AI and machine learning. He has good experience in designing end-to-end solutions and architecture for enterprise products.
Page
10
xiii Introduction When I was growing up, I was very fond of solving puzzles and reading detective stories. I would spend hours scratching my head about hidden treasure troves or solving a murder mystery on a lazy summer afternoon. I would picture myself inside the scene of action trying to catch the culprit red-handed from a vantage point in the attic. The characters would stay with me as I juggled around with the various possible scenarios and their equally plausible outcomes. Sometimes I would deduce the correct answer, mostly when I could see a pattern (or a motive), and be overjoyed with my feat. However, there were times when I would simply fail to come up with any solution and those were the times when I wished I had some superpowers. Fast-forward to today where I am working on keeping a critical business application alive when suddenly all hell breaks loose. The application is down, customers are not able to access anything and the business is incurring losses by the second. What went wrong? Don the detective’s hat, get your spy glasses, and try to find out the answer. This is similar to searching for the missing necklace from the queen’s jewelry box, albeit infinitely more complex. There are many characters involved, some with motives and even some with no motive at all. Finding a pattern or juggling scenarios in your head may not be the right start to your investigations. What you need is a superpower. Voila! This book is an exercise to provide you with that superpower to accurately find answers to the most critical questions when things start to go haywire with your application. You no longer need to rely on guesswork to find out what went wrong, nor do you need to use trial and error to fix a problem. We will help you get started with the concepts of monitoring, introduce you to popular open source monitoring tools, and help with finding the correct set of use cases for their implementation. The book covers in-depth technical details of open source software used in modern monitoring systems that are tailor-made for environments running microservices. This book is divided into two parts. Part 1 starts with an introduction to cloud native applications and the foundational concepts of monitoring. It then walks you through the various aspects of monitoring containerized workloads using Kubernetes as the de facto orchestration platform. We will dive deep into the architecture of a modern monitoring system and look at its individual components in detail. Part 2 will introduce you to
Page
11
xiv popular open source tools like Prometheus, TICK Stack, and Grafana, which are used by enterprises and startups alike and are well established as their tools of choice. After reading this book, you will have a much better understanding of the key terminology and general concepts around observability. You will learn about the complete spectrum of open source monitoring solutions available for applications, microservices and containers and how to quickly reap the benefits. Armed with this knowledge, you will be able to lead day-to-day operations more confidently and be better equipped to navigate the mesh of services sprawling in your environment. InTroduCTIon
Page
12
PART I Architecture of Modern Monitoring Systems for Cloud Native Applications Modern day systems are dynamic and complicated. They are highly distributed, ephemeral in nature but are built to sustain failure. This change in the underlying design of systems has led to introduction of orchestration platforms like Kubernetes which have abstracted away some of the regular concerns like health-checks, auto-remediation, auto scaling and load-balancing. Rather than looking for individual system behaviour one now must focus on the inter connection of these systems and understand their behaviour as a whole. It becomes necessary to gain better visibility into these systems, in order to learn about the performance of our services in production, which in turn provides with the feedback to build better and robust applications. In Part 1 of this book, we will introduce you to the two fundamental changes – Microservices and Containers, and their impact on modern software development and resulting application performance. We will talk about monitoring and discuss the impact of modern applications on traditional monitoring, while learning about the various terminologies and roles which have evolved to support this new paradigm of Cloud Native Applications. We would learn about Observability and how is it different from Monitoring and discuss in detail about the 3 pillars of Observability. We will also look at the individual components of monitoring, understand their functions and how they fit into the overall Modern Monitoring System. We will design an end-to-end architecture that is capable of identification of the minutest problems affecting your systems and then alerting you in the shortest possible time. By the end of Part 1 of this book, the reader will have a solid understanding of monitoring in general, its components and frameworks, the design of what a monitoring system should include and the associated terms and concepts.
Page
13
3 © Mainak Chakraborty and Ajit Pratap Kundan 2021 M. Chakraborty and A. P. Kundan, Monitoring Cloud-Native Applications, https://doi.org/10.1007/978-1-4842-6888-9_1 CHAPTER 1 Introduction to Modern Monitoring Digital transformation and cloud computing have changed the IT landscape of several organizations over the course of the last few years. As part of these initiatives, most companies have either already completed or are in the process of migrating their existing applications to the cloud. The reasons for this transition to a cloud-first approach can be attributed to on-demand scalability, pay-per-use model, and easy access to an unfathomably rich set of services previously unviable for organizations with access to limited resources. Garage startups today, even if bootstrapping, have the same platform available to build their business that is powering big commercial enterprises. In fact, many successful startups like Airbnb, Uber, and Netflix took the first-mover advantage of the cloud and are now leaders in their respective segments. The ongoing shift toward cloud and digital transformation has brought two fundamental changes—one in the field of application development and another in the field of infrastructure. These two significant changes have reshaped the way we interact digitally, for example, making payments using digital payment systems or mobile wallets, using IoT-enabled sensors in our smart homes, or getting a degree from an online classroom program. The efficacy with which these modern applications can handle millions of concurrent transactions today is feasible due to the evolution of the systems running them. This book is an introductory guide to modern-day monitoring of cloud native applications. It not only covers the technical details of how monitoring solutions work with microservices but also explains in detail the architecture, technical design, and steps for their implementation.
Page
14
4 In this particular chapter, we will introduce you to two fundamental changes in modern software development, the advent of microservices and containers and their impact on application deployment and performance. Then we will talk about monitoring and discuss the influence of modern applications on traditional monitoring. We will also learn about the various terminologies and roles which have evolved to support this new paradigm of cloud native applications. In addition to that, we will look at the various types of monitoring, the difference between reactive and proactive monitoring and the difference between push and pull monitoring. We will also discuss Kubernetes, its various components, and their respective functions. By the end of this chapter, you would have a good overview of modern monitoring and the associated concepts. Microservices Traditionally, applications were monolithic in nature. They were built on a single code base and deployed in a three-tier architecture of UI (Web), logic (app), and database (storage). In order to change an existing feature, introduce a new feature, or fix a bug, the complete code would need to be redeployed in production after fixing only that small portion of code which warranted changes. This would usually result in planned downtime for code redeployment, unplanned downtime if the code breaks in production and a massive effort between the development and operations teams to get it off the ground. This would lead to inevitable delays and subsequent revenue loss for the business. In fact, at one time, a release cycle of more than six months was considered as standard in the industry. As a direct result of digital transformation, businesses today want a quicker release cycle, which would mean rolling out newer features and capturing market share before the competition does. In order to speed up the release cycle, it became obvious to break the single monolithic application code block into smaller chunks of code, which can be worked upon more easily than the entire code base. The application functionality is split into several independent services called microservices, which provide particular functionalities within the overall application. Chapter 1 IntroduCtIon to Modern MonItorIng
Page
15
5 For example, imagine your favorite online retailer’s website running on monolithic architecture, as seen in Figure 1-1, having inventory management and payment systems in one single code base and deployed on premises. To introduce a change in the inventory management system, it would require changing only the relevant code but deploying the complete application again with downtime, which would result in unhappy customers and lost revenue. In contrast, if it were running on microservices architecture, only the microservice running the inventory management system would need to be modified and redeployed, and the rest of the application would still be accessible. This approach not only provides a newer set of services faster to your end users but also reduces overall downtime. Microservices is a modern approach to software development that structures an application as a collection of loosely coupled, independently deployable sets of services which are often developed, deployed, and maintained by a single team. This helps in rapid and frequent rollout of features even in a setup running hundreds of such services. Nowadays, born-in-cloud companies have release cycles which can be as frequent as once and even multiple times daily. This means businesses can provide added functionalities faster to their audience and thus generate more revenue or respond faster to customer concerns and therefore retain the existing customer base. Figure 1-1. Difference between monolithic and microservices architecture Chapter 1 IntroduCtIon to Modern MonItorIng
Page
16
6 Containers Containers provide an abstraction layer that helps developers and DevOps teams to develop, deploy, and run applications on common underlying resources, while still keeping them isolated. This means that an application can be packaged as a container along with its runtime environment such as libraries, binaries, configuration files, and other dependencies. Containers help developers to be more agile as they can now focus on the business logic rather than thinking about the environmental dependencies of their code. As shown in Figure 1-2, with regards to providing abstraction and isolation, containers closely resemble virtual machines (VMs); however unlike VMs, in a container the application is packaged with only the essential elements for it to run and not the entire guest OS. This allows containers to have unique capabilities when compared with VMs like quick spin-up, minimum overhead, miniscule footprint, and high portability. More and more developers today are consuming cloud-based container services, either Figure 1-2. Difference between VMs and containers Chapter 1 IntroduCtIon to Modern MonItorIng
Page
17
7 managed or unmanaged, to test and deploy their applications. This has given rise to the concept of “cloud native,” which is a term that broadly encompasses containerized environments that are run natively in cloud. As per the Cloud Native Computing Foundation (CNCF): Cloud native technologies empower organizations to build and run scalable applications in modern, dynamic environments such as public, private, and hybrid clouds. Containers, service meshes, microservices, immutable infra- structure, and declarative APIs exemplify this approach. These techniques enable loosely coupled systems that are resilient, manageable, and observ- able. Combined with robust automation, they allow engineers to make high- impact changes frequently and predictably with minimal toil. Containers have been around for some time but have really gained traction now along with the shift to microservices architecture. Companies who want to migrate their workloads easily from one environment to another can rely on containers to provide a logical abstraction of the application from the environment underneath, therefore supporting the use case of easy migration. Running servers (or VMs) is akin to having pets where each server is given a unique name and taken care of (manually managed). It is of grave concern if one server stops working and all necessary steps are taken to ensure that the server is healthy again (patching, updating, etc.). On the other hand, containers are ephemeral and in contrast to the philosophy of having pets, they are treated as cattle, raised to fulfill certain functions. If they were not to run anymore, nobody would care. They can simply be killed and replaced with another container. The important thing to note here is that containers are created keeping failure scenarios in mind. A microservices implementation doesn’t demand the exclusive use of containers; however, using them is the easiest way to run a microservices architecture on the cloud. Since containers can share resources in the same operating system instance with other application components, they can help in achieving better server utilization rates. Containers also spin up quickly and hence are better suited to respond to increasing demands from erratic workloads. The key attributes of elastic infrastructure and highly distributed application components are fueling the journey toward cloud native. However, these same attributes pose a unique challenge as containers can add significant operational complexity due to their short lifespan and highly dynamic nature. The network flow and routing in between the microservices needs to be configured and controlled. Chapter 1 IntroduCtIon to Modern MonItorIng
Page
18
8 In scenarios where the number of containers is lower, one can manually perform these operational tasks, but with rapid adoption, the number of containers quickly starts to grow in production and the need for container orchestration starts becoming obvious. Container orchestrators provide management tasks such as resource allocation and management, automation, scaling, health checks, networking, load balancing, and so on. Container orchestration tools like Kubernetes help in automating the deployment, scaling and management of containerized workloads across cloud environments. We will look at Kubernetes in detail later in this chapter. Monitoring Monitoring refers to the complete process of gaining visibility and meaningful insight into the state of a system. It includes building a well-defined system of measurement to verify the intended behavior of an entity and quickly notify the administrators involved if there is any drift from that behavior. It helps in identification of the causes of anomalous behavior and rectification of those issues to avoid any potential reoccurrences. As defined by Rob Ewaschuk in Site Reliability Engineering: How Google Runs Production Systems (O’Reilly, 2016): Monitoring means collecting, processing, aggregating and displaying real- time quantitative data about a system, such as query counts and types, error counts and types, processing times, and server lifetimes. Importance of Monitoring Monitoring is important to identify capacity usage and performance trends over a particular time period. It helps in finding the historical utilization of resources, thereby assisting in making informed decisions regarding the future procurement of capacity and it also serves as an input to architectural redesign and most importantly, provides incredible insight into the workings of a system. It is also useful to preemptively discover anomalous behavior before they develop into pesky problems, thereby helping in sustaining availability and maintaining service quality. Chapter 1 IntroduCtIon to Modern MonItorIng
Page
19
9 Systems emit data either at a regular interval automatically or when an event occurs. By leveraging this data with the help of suitable monitoring systems, we can - 1. Get proactively alerted of potential problems in the system, 2. Quickly analyze and remediate a problem which has already occurred, 3. Determine the overall health of the environment Therefore, capturing this data is important for the efficient running of your systems and applications. The main use case of monitoring is to timely identify the source of a problem. During an outage, time is of essence, as everyone is geared towards bringing the system up, but there is not much time for a detailed analysis. To resolve this, monitoring systems need to process huge datasets of complex information and present them effectively in easy-to-remember data points. Based on this analysis, operators can then get to the bottom of the problem and rectify it quickly. The techniques used in modern-day monitoring are varied and cut across the fields of real-time data processing and statistical data analysis. Data visualization also plays an important role, as the processed data must be meaningfully displayed in a human- readable format. There are many ways to interpret data, and monitoring data gives you many different perspectives on the underlying issues from the same dataset. DevOps—Roles and User Personas Before the era of DevOps, developers would usually write their code with very little concern for how it will perform in production and push it over the wall to the operations team. The operations team, which was responsible for keeping the application up and running, would invariably point fingers at developers when the code would break in production. DevOps emerged as an answer to resolve this conflict of misaligned priorities between the two groups and proposed a new set of practices to bridge the gap between software development and software operations teams. This evolution of DevOps is depicted in Figure 1-3. Chapter 1 IntroduCtIon to Modern MonItorIng
Page
20
10 To align with the principles of DevOps and Agile methodology, companies have created specific roles within their organizations. One of these roles, which evolved at Google, is that of site reliability engineers (SREs), who are responsible for keeping the systems up and running. SREs monitor their systems for meeting predefined service- level objectives (SLOs) by • Providing information on system behavior • Identifying trends of system usage and performance • Notifying and alerting on outliers and anomalies • Diagnosing the problem SREs are tasked with the reliability of the production system and therefore need to have a solid understanding of the services running in production and how are they being monitored. If unarmed with proper tools and information, SREs won’t be able to quickly locate important information even after identifying abnormal behavior. Figure 1-4 shows the five pillars of DevOps and their corresponding SRE practices: Figure 1-3. Evolution of DevOps Chapter 1 IntroduCtIon to Modern MonItorIng