Machine Learning For Dummies®, IBM Limited Edition (Judith Hurwitz Daniel Kirsch) (Z-Library)

These materials are © 2018 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited. Machine Learning IBM Limited Edition by Judith Hurwitz and Daniel Kirsch

These materials are © 2018 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited. Machine Learning For Dummies®, IBM Limited Edition Published by John Wiley & Sons, Inc. 111 River St. Hoboken, NJ 07030-5774 www.wiley.com Copyright © 2018 by John Wiley & Sons, Inc. No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without the prior written permission of the Publisher. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley.com/go/permissions. Trademarks: Wiley, For Dummies, the Dummies Man logo, The Dummies Way, Dummies.com, Making Everything Easier, and related trade dress are trademarks or registered trademarks of John Wiley & Sons, Inc. and/or its affiliates in the United States and other countries, and may not be used without written permission. IBM and the IBM logo are registered trademarks of International Business Machines Corporation. All other trademarks are the property of their respective owners. John Wiley & Sons, Inc., is not associated with any product or vendor mentioned in this book. LIMIT OF LIABILITY/DISCLAIMER OF WARRANTY: THE PUBLISHER AND THE AUTHOR MAKE NO REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE ACCURACY OR COMPLETENESS OF THE CONTENTS OF THIS WORK AND SPECIFICALLY DISCLAIM ALL WARRANTIES, INCLUDING WITHOUT LIMITATION WARRANTIES OF FITNESS FOR A PARTICULAR PURPOSE. NO WARRANTY MAY BE CREATED OR EXTENDED BY SALES OR PROMOTIONAL MATERIALS.  THE ADVICE AND STRATEGIES CONTAINED HEREIN MAY NOT BE SUITABLE FOR EVERY SITUATION. THIS WORK IS SOLD WITH THE UNDERSTANDING THAT THE PUBLISHER IS NOT ENGAGED IN RENDERING LEGAL, ACCOUNTING, OR OTHER PROFESSIONAL SERVICES.  IF PROFESSIONAL ASSISTANCE IS REQUIRED, THE SERVICES OF A COMPETENT PROFESSIONAL PERSON SHOULD BE SOUGHT. NEITHER THE PUBLISHER NOR THE AUTHOR SHALL BE LIABLE FOR DAMAGES ARISING HEREFROM. THE FACT THAT AN ORGANIZATION OR WEBSITE IS REFERRED TO IN THIS WORK AS A CITATION AND/OR A POTENTIAL SOURCE OF FURTHER INFORMATION DOES NOT MEAN THAT THE AUTHOR OR THE PUBLISHER ENDORSES THE INFORMATION THE ORGANIZATION OR WEBSITE MAY PROVIDE OR RECOMMENDATIONS IT MAY MAKE.  FURTHER, READERS SHOULD BE AWARE THAT INTERNET WEBSITES LISTED IN THIS WORK MAY HAVE CHANGED OR DISAPPEARED BETWEEN WHEN THIS WORK WAS WRITTEN AND WHEN IT IS READ. For general information on our other products and services, or how to create a custom For Dummies book for your business or organization, please contact our Business Development Department in the U.S. at 877-409-4177, contact info@dummies.biz, or visit www.wiley.com/go/custompub. For information about licensing the For Dummies brand for products or services, contact BrandedRights&Licenses@Wiley.com. ISBN: 978-1-119-45495-3 (pbk); ISBN: 978-1-119-45494-6 (ebk) Manufactured in the United States of America 10 9 8 7 6 5 4 3 2 1 Publisher’s Acknowledgments Some of the people who helped bring this book to market include the following: Project Editor: Carrie A. Burchfield Editorial Manager: Rev Mengle Acquisitions Editor: Steve Hayes Business Development Representative: Sue Blessing IBM Contributors: Jean-Francois Puget, Nancy Hensley, Brad Murphy, Troy Hernandez

Table of Contents iii These materials are © 2018 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited. Table of Contents INTRODUCTION ............................................................................................... 1 About This Book ................................................................................... 1 Foolish Assumptions ............................................................................ 2 Icons Used in This Book ....................................................................... 2 CHAPTER 1: Understanding Machine Learning ................................. 3 What Is Machine Learning? ................................................................. 4 Iterative learning from data ........................................................... 5 What’s old is new again .................................................................. 5 Defining Big Data .................................................................................. 6 Big Data in Context with Machine Learning ...................................... 7 The Need to Understand and Trust your Data ................................. 8 The Importance of the Hybrid Cloud ................................................. 9 Leveraging the Power of Machine Learning ..................................... 9 Descriptive analytics ..................................................................... 10 Predictive analytics ....................................................................... 10 The Roles of Statistics and Data Mining with Machine Learning ............................................................................... 11 Putting Machine Learning in Context .............................................. 12 Approaches to Machine Learning .................................................... 14 Supervised learning ...................................................................... 15 Unsupervised learning ................................................................. 15 Reinforcement learning ............................................................... 16 Neural networks and deep learning ........................................... 17 CHAPTER 2: Applying Machine Learning .............................................. 19 Getting Started with a Strategy......................................................... 19 Using machine learning to remove biases from strategy ........ 20 More data makes planning more accurate ............................... 22 Understanding Machine Learning Techniques ............................... 22 Tying Machine Learning Methods to Outcomes ............................ 23 Applying Machine Learning to Business Needs.............................. 23 Understanding why customers are leaving ............................... 24 Recognizing who has committed a crime .................................. 25 Preventing accidents from happening ....................................... 26

iv Machine Learning For Dummies, IBM Limited Edition These materials are © 2018 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited. CHAPTER 3: Looking Inside Machine Learning ................................ 27 The Impact of Machine Learning on Applications .......................... 28 The role of algorithms .................................................................. 28 Types of machine learning algorithms ....................................... 29 Training machine learning systems ............................................ 33 Data Preparation ................................................................................ 34 Identify relevant data ................................................................... 34 Governing data .............................................................................. 36 The Machine Learning Cycle ............................................................. 37 CHAPTER 4: Getting Started with Machine Learning ................. 39 Understanding How Machine Learning Can Help .......................... 39 Focus on the Business Problem ....................................................... 40 Bringing data silos together ........................................................ 41 Avoiding trouble before it happens ............................................ 42 Getting customer focused ........................................................... 43 Machine Learning Requires Collaboration ...................................... 43 Executing a Pilot Project .................................................................... 44 Step 1: Define an opportunity for growth .................................. 44 Step 2: Conducting a pilot project ............................................... 44 Step 3: Evaluation ......................................................................... 45 Step 4: Next actions ...................................................................... 45 Determining the Best Learning Model ............................................ 46 Tools to determine algorithm selection ..................................... 46 Approaching tool selection .......................................................... 47 CHAPTER 5: Learning Machine Skills ....................................................... 49 Defining the Skills That You Need .................................................... 49 Getting Educated ................................................................................ 53 IBM-Recommended Resources ........................................................ 56 CHAPTER 6: Using Machine Learning to Provide Solutions to Business Problems .................................... 57 Applying Machine Learning to Patient Health ................................ 57 Leveraging IoT to Create More Predictable Outcomes.................. 58 Proactively Responding to IT Issues ................................................. 59 Protecting Against Fraud ................................................................... 60 CHAPTER 7: Ten Predictions on the Future of Machine Learning ............................................................... 63

These materials are © 2018 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited. Introduction Machine learning is having a dramatic impact on the way software is designed so that it can keep pace with busi- ness change. Machine learning is so dramatic because it helps you use data to drive business rules and logic. How is this different? With traditional software development models, pro- grammers wrote logic based on the current state of the business and then added relevant data. However, business change has become the norm. It is virtually impossible to anticipate what changes will transform a market. The value of machine learning is that it allows you to continually learn from data and predict the future. This powerful set of algo- rithms and models are being used across industries to improve processes and gain insights into patterns and anomalies within data. But machine learning isn’t a solitary endeavor; it’s a team process that requires data scientists, data engineers, business analysts, and business leaders to collaborate. The power of machine learn- ing requires a collaboration so the focus is on solving business problems. About This Book Machine Learning For Dummies, IBM Limited Edition, gives you insights into what machine learning is all about and how it can impact the way you can weaponize data to gain unimaginable insights. Your data is only as good as what you do with it and how you manage it. In this book, you discover types of machine learn- ing techniques, models, and algorithms that can help achieve results for your company. This information helps both business and technical leaders learn how to apply machine learning to anticipate and predict the future. Introduction 1

2 Machine Learning For Dummies, IBM Limited Edition These materials are © 2018 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited. Foolish Assumptions The information in this book is useful to many people, but we have to admit that we did make a few assumptions about who we think you are: » You’re already familiar with how machine learning algo- rithms are being used within your organization to create new software. You need to be prepared to lead your team in the right direction so that the company gains maximum value from the use of these powerful algorithms and models. » You’re planning a long-term strategy to create software that can stand the test of time. Management wants to be able to leverage all the important data about customers, employees, prospects, and business trends. Your goal is to be prepared for the future. » You understand the huge potential value of the data that exists throughout your organization. » You understand the benefits of machine learning and its impact on the company, and you want to make sure that your team is ready to apply this power to remain competitive as new business models emerge. » You’re a business leader who wants to apply the most important emerging technologies to be as creative and innovative as possible. Icons Used in This Book The following icons are used to point out important information throughout the book: Tips help identify information that needs special attention. These icons point out content that you should pay attention to. We highlight common pitfalls in taking advantage of machine learn- ing models and algorithms. This icon highlights important information that you should remember.

CHAPTER 1 Understanding Machine Learning 3 These materials are © 2018 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited. Understanding Machine Learning Machine learning, artificial intelligence (AI), and cognitive computing are dominating conversations about how emerging advanced analytics can provide businesses with a competitive advantage to the business. There is no debate that existing business leaders are facing new and unanticipated competitors. These businesses are looking at new strategies that can prepare them for the future. While a business can try different strategies, they all come back to a fundamental truth — you have to follow the data. In this chapter, we delve into what the value of machine learning can be to your business strategy. How should you think about machine learning? What can you offer the busi- ness based on advanced analytics technique that can be a game-changer? Chapter 1 IN THIS CHAPTER » Defining machine learning and big data » Trusting your data » Looking at why the hybrid cloud is important » Using machine learning and artificial intelligence » Understanding the approaches to machine learning

4 Machine Learning For Dummies, IBM Limited Edition These materials are © 2018 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited. What Is Machine Learning? Machine learning has become one of the most important topics within development organizations that are looking for innovative ways to leverage data assets to help the business gain a new level of understanding. Why add machine learning into the mix? With the appropriate machine learning models, organizations have the ability to continually predict changes in the business so that they are best able to predict what’s next. As data is constantly added, the machine learning models ensure that the solution is constantly updated. The value is straightforward: If you use the most appropriate and constantly changing data sources in the context of machine learning, you have the opportunity to predict the future. Machine learning is a form of AI that enables a system to learn from data rather than through explicit programming. However, machine learning is not a simple process. Machine learning uses a variety of algorithms that iteratively learn from data to improve, describe data, and predict outcomes. As the algorithms ingest training data, it is then possible to pro- duce more precise models based on that data. A machine learn- ing model is the output generated when you train your machine learning algorithm with data. After training, when you provide a model with an input, you will be given an output. For example, a predictive algorithm will create a predictive model. Then, when you provide the predictive model with data, you will receive a pre- diction based on the data that trained the model. Machine learn- ing is now essential for creating analytics models. You likely interact with machine learning applications without realizing. For example, when you visit an e-commerce site and start viewing products and reading reviews, you’re likely pre- sented with other, similar products that you may find interesting. These recommendations aren’t hard coded by an army of devel- opers. The suggestions are served to the site via a machine learn- ing model. The model ingests your browsing history along with other shoppers’ browsing and purchasing data in order to present other similar products that you may want to purchase.

CHAPTER 1 Understanding Machine Learning 5 These materials are © 2018 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited. Iterative learning from data Machine learning enables models to train on data sets before being deployed. Some machine learning models are online and contin- uously adapt as new data is ingested. On the other hand, other models, called offline machine learning models, are derived from machine learning algorithms but, once deployed, do not change. This iterative process of online models leads to an improvement in the types of associations that are made between data elements. Due to their complexity and size, these patterns and associations could have easily been overlooked by human observation. After a model has been trained, these models can be used in real time to learn from data. In addition, complex algorithms can be automatically adjusted based on rapid changes in variables, such as sensor data, time, weather data, and customer sentiment metrics. For example, inferences can be made from a machine learning model — if the weather changes quickly, a weather predicting model can predict a tornado, and a warning siren can be triggered. The improve- ments in accuracy are a result of the training process and auto- mation that is part of machine learning. Online machine learning algorithms continuously refine the models by continuously pro- cessing new data in near real time and training the system to adapt to changing patterns and associations in the data. What’s old is new again AI and machine learning algorithms aren’t new. The field of AI dates back to the 1950s. Arthur Lee Samuels, an IBM researcher, developed one of the earliest machine learning programs  — a self-learning program for playing checkers. In fact, he coined the term machine learning. His approach to machine learning was explained in a paper published in the IBM Journal of Research and Development in 1959. Over the decades, AI techniques have been widely used as a method of improving the performance of underlying code. In the last few years with the focus on distributed computing models and cheaper compute and storage, there has been a surge of inter- est in AI and machine learning that has lead to a huge amount of money being invested in startup software companies. Today, we

6 Machine Learning For Dummies, IBM Limited Edition These materials are © 2018 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited. are seeing major advancements and commercial solutions. Why has the market become real? There are six key enablers: » Modern processors have become increasingly powerful and increasingly dense. The density to performance ratio has improved dramatically. » The cost of storing and managing large amounts of data has been dramatically lowered. In addition, new storage innovations have led to faster performance and the ability to analyze vastly larger data sets. » The ability to distribute compute processing across clusters of computers has dramatically improved the ability to analyze complex data in record time. » There are more commercial data sets available to support analytics, including weather data, social media data, and medical data sets. Many of these are available as cloud services and well-defined Application Programming Interfaces (APIs). » Machine learning algorithms have been made available through open-source communities with large user bases. Therefore, there are more resources, frameworks, and libraries that have made development easier. » Visualization has gotten more consumable. You don’t need to be a data scientist to interpret results, making use of machine learning broader within many industries. Defining Big Data Big data is any kind of data source that has at least one of four shared characteristics, called the four Vs: » Extremely large Volumes of data » The ability to move that data at a high Velocity of speed » An ever-expanding Variety of data sources » Veracity so that data sources truly represent truth The accuracy of a machine learning model can increase substan- tially if it’s trained on big data. Without enough data, you are

CHAPTER 1 Understanding Machine Learning 7 These materials are © 2018 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited. trying to make decisions on small subsets of your data that might lead to misinterpreting a trend or missing a pattern that is just starting to emerge. While big data can be very useful for training machine learning models, organizations can use machine learn- ing with just a few thousand data points. Don’t underestimate the task at hand. Data must be able to be verified based on both accuracy and context. An innovative busi- ness in a fast-changing market will want to deploy a model that can make inferences in milliseconds to quickly assess the best offer for an at-risk customer to keep her happy. It is necessary to identify the right amount and types of data that can be analyzed to impact business outcomes. Big data incorporates all data, includ- ing structured, unstructured, and semi-structured data from email, social media, text streams, images, and machine sensors. Traditional Business Intelligence (BI) products weren’t really designed to handle the complexities of constantly changing data sources. BI tools are typically designed to work with highly structured, well-understood data, often stored in a relational data repository. These traditional BI tools typically only analyze snapshots of data rather than the entire data set. Analytics on big data requires technology designed to gather, store, manage, and manipulate vast amounts data at the right speed and at the right time to gain the right insights. With the evolution of comput- ing technology and the emergence of hybrid cloud architectures, it’s now possible to manage immense volumes of data that previ- ously could have only been handled by supercomputers at great expense. Big Data in Context with Machine Learning Machine learning requires the right set of data that can be applied to a learning process. An organization does not have to have big data in order to use machine learning techniques; however, big data can help improve the accuracy of machine learning models. With big data, it is now possible to virtualize data so it can be stored in the most efficient and cost-effective manner whether on- premises or in the cloud. In addition, improvements in network speed and reliability have removed other physical limitations of

8 Machine Learning For Dummies, IBM Limited Edition These materials are © 2018 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited. being able to manage massive amounts of data at the acceptable speed. Add to this the impact of changes in the price and sophis- tication of computer memory, and with all these technology tran- sitions, it’s now possible to imagine how companies can leverage data in ways that would’ve been inconceivable only five years ago. No technology transition happens in isolation; change happens when there is an unsolved business problem combined with the maturation of technology. There are countless examples of important technologies that have matured enough to support the renaissance of machine learning. These maturing big data tech- nologies include data virtualization, parallel processing, distrib- uted file systems, in-memory databases, containerization, and micro-services. This combination of technology advances can help organizations address significant business problems. Busi- nesses have never lacked large amounts of data. Leaders have been frustrated for decades about their inability to use the rich- ness of data sources to gain actionable insights from their data. Armed with big data technologies and machine learning models, organizations are able to anticipate the future and be better pre- pared for disruption. The Need to Understand and Trust your Data It is not enough to simply ingest vast amounts of data. Providing accurate machine learning models requires that the source data be accurate and meaningful. In addition, these data sources are meaningful when combined with each other so that the model is accurate and trusted. You have to understand the origin of your data sources and whether they make sense when they’re combined. In addition to trusting your data, it also important to perform data cleansing or tidying. Cleaning data means that you transform your data into a form that can be understood by a machine learn- ing algorithm. For example, algorithms use numbers, but data is often in the form of words. You have to turn those words into numbers. In addition, you have to make sure those numbers are

CHAPTER 1 Understanding Machine Learning 9 These materials are © 2018 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited. sensibly derived and internally consistent. You need to decide how you handle missing data and other data irregularities. Data refinement provides the foundation for building analyti- cal models that deliver results you can trust. The process of data refinement will help to ensure that your data is timely, clean, and well understood. The Importance of the Hybrid Cloud When approaching machine learning and big data, many organi- zations have discovered that a combination of public and private cloud services is the most pragmatic way to ensure scalability, security, and compliance. To deepen learning, a company may, for example, want to leverage Graphics Processing Units (GPUs) on the cloud rather than building their own GPU-based environ- ment. This is a hybrid approach. A hybrid cloud is a combination of on-premises and public cloud services intended to work in unison. The hybrid environment provides businesses with the flexibility to select the most appro- priate service for specific workloads based on critical factors such as cost, security, and performance. Cloud computing allows businesses to test new endeavors with- out the large upfront costs of on-premises hardware. Rather than going through procurement and integration, teams can imme- diately begin working with machine learning techniques. As the organization matures, it may choose to bring some of the hard- ware on-premises because of security and control or the cloud computing costs that can quickly escalate. Leveraging the Power of Machine Learning The role of analytics in an organization’s operational processes has changed significantly over the past 30 years. Companies are expe- riencing a progression in analytics maturity levels ranging from descriptive analytics to predictive analytics to machine learn- ing and cognitive computing. Companies have been successful at

10 Machine Learning For Dummies, IBM Limited Edition These materials are © 2018 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited. using analytics to understand both where they’ve been and how they can learn from the past to anticipate the future. They are able to describe how various actions and events will impact outcomes. While the knowledge from this analysis can be used to make pre- dictions, typically these predictions are made through a lens of preconceived expectations. Data scientists and business analysts have been constrained to make predictions based on analytical models that are based on historical data. However, there are always unknown factors that can have a significant impact on future outcomes. Companies need a way to build predictive models that can react and change when there are changes to the business environment. In this section, we give you two types of approaches to advanced analytics. Descriptive analytics Descriptive analytics helps the analysts understand current reality in the business. You need to understand the context for historical data in order to understand the current reality of where the busi- ness is today. This approach helps an organization answer ques- tions such as which product styles are selling better this quarter as compared to last quarter, and which regions are exhibiting the highest/lowest growth. Predictive analytics Predictive analytics helps anticipate changes based on under- standing the patterns and anomalies within that data. With this model, the analyst assimilates a number of related data sources in order to predict outcomes. Predictive analytics leverages sophisti- cated machine learning algorithms to gain ongoing insights. A predictive analytics tool requires that the model is con- stantly provided with new data that reflects business change. This approach improves the ability of the business to anticipate subtle changes in customer preferences, price erosion, market changes, and other factors that will impact the future of business outcomes. With a predictive model, you look into the future. For example, you can answer the following types of questions:

CHAPTER 1 Understanding Machine Learning 11 These materials are © 2018 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited. » How can the web experience be transformed to entice a customer to buy frequently? » How do you predict how a stock or a portfolio will perform based on international news and internal financial factors? » Which combination of drugs will provide the best outcome for this cancer patient based on the specific characteristics of the tumor and genetic sequencing? The Roles of Statistics and Data Mining with Machine Learning The disciplines of statistics, data mining, and machine learning all have a role in understanding data, describing the character- istics of a data set and finding relationships and patterns in that data to build a model. There is a great deal of overlap in how the techniques and tools of these disciplines are applied to solving business problems. Many of the widely used data mining and machine learning algo- rithms are rooted in classical statistical analysis. Data scientists combine technology backgrounds with expertise in statistics, data mining, and machine learning to use all disciplines in collabo- ration. Regardless of the combination of capabilities and tech- nology used to predict outcomes, having an understanding of the business problem, business goals, and subject matter expertise is essential. You can’t expect to get good results by focusing on the statistics alone without considering the business side. The following points highlight how these capabilities relate to each other: » Statistics is the science of analyzing the data. Classical or conventional statistics is inferential in nature, meaning it’s used to reach conclusions about the data (various param- eters). Statistical modeling is focused primarily on making inferences and understanding the characteristics of the variables. Machine learning models leverage statistical algorithms and apply them to predict analytics. In a statistical model, a hypothesis is a testable way to confirm the validity of the specific algorithm.

12 Machine Learning For Dummies, IBM Limited Edition These materials are © 2018 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited. » Data mining, which is based on the principles of statistics, is the process of exploring and analyzing large amounts of data to discover patterns in that data. Algorithms are used to find relationships and patterns in the data, and then this information about the patterns is used to make forecasts and predictions. Data mining is used to solve a range of business problems, such as fraud detection, market basket analysis, and customer churn analysis. Traditionally, organizations use data mining tools on large volumes of structured data, such as customer relationship management databases or aircraft parts inventories. The goal of data mining is to explain and understand the data. Data mining is not intended to make predictions or back up hypotheses. Some analytics vendors provide software solutions that enable data mining of a combination of structured and unstructured data. Generally, the goal of the data mining is to extract data from a larger data set for the purposes of classification or prediction. In data mining, data is clustered into groups. For example, a marketer might be interested in the characteristics of people who responded to a promo- tional offer versus those who didn’t respond to the promo- tion. In this example, data mining would be used to extract the data according to the two different classes and analyze the characteristics of each class. A marketer might be interested in predicting those who will respond to a promo- tion. Data mining tools are intended to support the human decision-making process. Therefore, data mining is intended to show patterns that can be used by humans. In contrast, machine learning automates the process of identifying patterns that are used to make predictions. Machine learning algorithms are covered in the next section, “Putting Machine Learning in Context,” in greater detail due to the importance of this discipline to advanced analytics. Putting Machine Learning in Context To understand the role of machine learning, we need to give you some context. AI, machine learning, and deep learning are all terms that are frequently mentioned when discussing big data, analytics, and advanced technology. AI can be understood as the

CHAPTER 1 Understanding Machine Learning 13 These materials are © 2018 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited. broadest way of describing systems that can “think.” For exam- ple, thermostats that learn your preference or applications that can identify people and what they are doing in photographs can be thought of as AI systems. As illustrated in Figure 1-1, there are four main subsets of AI. In this book, we focus on machine learning. However, in order to understand machine learning, it is important to put it in perspective. When we explore machine learning, we focus on the ability to learn and adapt a model based on the data rather than explicit programming. In Chapter 6, we focus on applying machine learn- ing to solving business problems. Before we delve into the types of machine learning, it is important to understand the other subsets of AI: » Reasoning: Machine reasoning allows a system to make inferences based on data. In essence, reasoning helps fill in the blanks when there is incomplete data. Machine reason- ing helps make sense of connected data. For example, if a system has enough data and is asked “What is a safe internal temperature for eating a drumstick?” the system would be capable of telling you that the answer is 165 degrees. The FIGURE 1-1: AI is the overall category that includes machine learning and natural language processing.

14 Machine Learning For Dummies, IBM Limited Edition These materials are © 2018 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited. logic chain would be as follows: A drumstick that is eaten (as opposed to a part of a musical instrument) refers to a chicken leg, a chicken leg contains dark chicken meat, dark chicken meat needs to be cooked at 165 degrees, therefore the answer is 165 degrees. Note: In this example, the system was never explicitly trained on the safe internal temperature of chicken drumsticks. Instead the system used the knowl- edge it had to fill in the data gaps. » Natural Language Processing (NLP): NLP is the ability to train computers to understand both written text and human speech. NLP techniques are needed to capture the meaning of unstructured text from documents or communication from the user. Therefore, NLP is the primary way that systems can interpret text and spoken language. NLP is also one of the fundamental technologies that allows non-technical people to interact with advanced technologies. For example, rather than needing to code, NLP can help users ask a system questions about complex data sets. Unlike structured database informa- tion that relies on schemas to add context and meaning to the data, unstructured information must be parsed and tagged to find the meaning of the text. Tools required for NLP include categorization, ontologies, tapping, catalogs, dictionaries, and language models. » Planning: Automated planning is the ability for an intelligent system to act autonomously and flexibly to construct a sequence of actions to reach a final goal. Rather than a pre-programmed decision-making process that goes from A to B to C to reach a final output, automated planning is complex and requires a system to adapt based on the context surrounding the given challenge. Approaches to Machine Learning Machine learning techniques are required to improve the accuracy of predictive models. Depending on the nature of the business problem being addressed, there are different approaches based on the type and volume of the data. In this section, we discuss the categories of machine learning.

CHAPTER 1 Understanding Machine Learning 15 These materials are © 2018 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited. Supervised learning Supervised learning typically begins with an established set of data and a certain understanding of how that data is classified. Supervised learning is intended to find patterns in data that can be applied to an analytics process. This data has labeled features that define the meaning of data. For example, there could be mil- lions of images of animals and include an explanation of what each animal is and then you can create a machine learning appli- cation that distinguishes one animal from another. By labeling this data about types of animals, you may have hundreds of cat- egories of different species. Because the attributes and the mean- ing of the data have been identified, it is well understood by the users that are training the modeled data so that it fits the details of the labels. When the label is continuous, it is a regression; when the data comes from a finite set of values, it known as classifica- tion. In essence, regression used for supervised learning helps you understand the correlation between variables. An example of supervised learning is weather forecasting. By using regression analysis, weather forecasting takes into account known historical weather patterns and the current conditions to provide a predic- tion on the weather. The algorithms are trained using preprocessed examples, and at this point, the performance of the algorithms is evaluated with test data. Occasionally, patterns that are identified in a subset of the data can’t be detected in the larger population of data. If the model is fit to only represent the patterns that exist in the training subset, you create a problem called overfitting. Overfit- ting means that your model is precisely tuned for your training data but may not be applicable for large sets of unknown data. To protect against overfitting, testing needs to be done against unforeseen or unknown labeled data. Using unforeseen data for the test set can help you evaluate the accuracy of the model in predicting outcomes and results. Supervised training models have broad applicability to a variety of business problems, including fraud detection, recommendation solutions, speech recognition, or risk analysis. Unsupervised learning Unsupervised learning is best suited when the problem requires a massive amount of data that is unlabeled. For example, social media applications, such as Twitter, Instagram, Snapchat, and