Machine Learning for Streaming Data with Python (Joos Korstanje) (z-library.sk, 1lib.sk, z-lib.sk)
Python
Rapidly build practical online machine learning solutions using River and other top key frameworks Apply machine learning to streaming data with the help of practical examples, and deal with challenges that surround streaming Key Features Work on streaming use cases that are not taught in most data science courses Gain experience with state-of-the-art tools for streaming data Mitigate various challenges while handling streaming data Book Description Streaming data is the new top technology to watch out for in the field of data science and machine learning. As business needs become more demanding, many use cases require real-time analysis as well as real-time machine learning. This book will help you to get up to speed with data analytics for streaming data and focus strongly on adapting machine learning and other analytics to the case of streaming data. You will first learn about the architecture for streaming and real-time machine learning. Next, you will look at the state-of-the-art frameworks for streaming data like River. Later chapters will focus on various industrial use cases for streaming data like Online Anomaly Detection and others. As you progress, you will discover various challenges and learn how to mitigate them. In addition to this, you will learn best practices that will help you use streaming data to generate real-time insights. By the end of this book, you will have gained the confidence you need to stream data in your machine learning models. What you will learn Understand the challenges and advantages of working with streaming data Develop real-time insights from streaming data Understand the implementation of streaming data with various use cases to boost your knowledge Develop a PCA alternative that can work on real-time data Explore best practices for handling streaming data that you absolutely need to remember Develop an API for real-time machine learning inference Who this book is for This book is for data scientists and mac
55
Views
Registered users can read the full content for free
Register as a Gaohf Library member to read the complete e-book online for free and enjoy a better reading experience.
Page
1
(This page has no text content)
Page
2
BIRMINGHAM—MUMBAI Machine Learning for Streaming Data with Python Copyright © 2022 Packt Publishing All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information. Publishing Product Manager: Dinesh Chaudhary Content Development Editor: Joseph Sunil Technical Editor: Rahul Limbachiya Copy Editor: Safis Editing Project Coordinator: Farheen Fathima Proofreader: Safis Editing Indexer: Sejal Dsilva Production Designer: Shankar Kalbhor Marketing Coordinator: Shifa Ansari and Abeer Riyaz Dawe
Page
3
First published: July 2022 Production reference: 1240622 Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK. ISBN 978-1-80324-836-3 www.packt.com Contributors About the author Joos Korstanje, with his master's degrees in both environmental sciences and data science, has been working on statistics and data science for almost 10 years. Through his work in different companies including Disney, AXA, and others, he has closely followed developments in data science and related fields. This experience in the business world has allowed him to write about data science from an applied point of view (through his books, Medium, Towards Data Science, LinkedIn, and more). About the reviewer Olivia Petris is a big data engineer working as an IT consultant in a technology and advisory services firm based in Paris. On her professional journey, she's always looking for challenging and interesting assignments. Since her engineering diploma in computer science, she has chosen to be in the data science and big data field. Therefore, she continues to improve her skills and keep up to date with new IT and technology developments. In her free time, she enjoys traveling, practicing karate, and hanging out with her family and friends.
Page
4
Table of Contents Preface
Page
5
Part 1: Introduction and Core Concepts of Streaming Data
Page
6
Chapter 1 : An Introduction to Streaming Data Technical requirements Setting up a Python environment A short history of data science Working with streaming data Streaming data versus batch data Advantages of streaming data Examples of successful implementation of streaming analytics Challenges of streaming data How to get started with streaming data Common use cases for streaming data Streaming versus big data Real-time data formats and importing an example dataset in Python Summary Further reading
Page
7
Chapter 2 : Architectures for Streaming and Real- Time Machine Learning Technical requirements Python environment Defining your analytics as a function Understanding microservices architecture Communicating between services through APIs Demystifying the HTTP protocol The GET request The POST request JSON format for communication between systems RESTful APIs Building a simple API on AWS API Gateway in AWS Lambda in AWS Data-generating process on a local machine Implementing the example More architectural considerations Other AWS services and other services in general that have the same functionality Big data tools for real time streaming Calling a big data environment in real time Summary Further reading
Page
8
Chapter 3 : Data Analysis on Streaming Data Technical requirements Python environment Descriptive statistics on streaming data Why are descriptive statistics different on streaming data? Introduction to sampling theory Comparing population and sample Population parameters and sample statistics Sampling distribution Sample size calculations and confidence level Rolling descriptive statistics from streaming Exponential weight Tracking convergence as an additional KPI Overview of the main descriptive statistics The mean The median The mode Standard deviation Variance Quartiles and interquartile range Correlations Real-time visualizations Opening the dashboard Comparing Plotly's Dash and other real-time visualization tools
Page
9
Building basic alerting systems Alerting systems on extreme values Alerting systems on process stability (mean and median) Alerting systems on constant variability (std and variance) Basic alerting systems using statistical process control Summary Further reading
Page
10
Part 2: Exploring Use Cases for Data Streaming
Page
11
Chapter 4 : Online Learning with River Technical requirements Python environment What is online machine learning? How is online learning different from regular learning? Advantages of online learning Challenges of online learning Types of online learning Using River for online learning Training an online model with River Improving the model evaluation Building a multiclass classifier using one-vs-rest Summary Further reading
Page
12
Chapter 5 : Online Anomaly Detection Technical requirements Python environment Defining anomaly detection Are outliers a problem? Exploring use cases of anomaly detection Fraud detection in financial institutions Anomaly detection on your log data Fault detection in manufacturing and production lines Hacking detection in computer networks (cyber security) Medical risks in health data Predictive maintenance and sensor data Comparing anomaly detection and imbalanced classification The problem of imbalanced data The F1 score SMOTE oversampling Anomaly detection versus classification Algorithms for detecting anomalies in River The use of thresholders in River anomaly detection Anomaly detection algorithm 1 – One-Class SVM Anomaly detection algorithm 2 – Half-Space-Trees Going further with anomaly detection Summary Further reading
Page
13
Chapter 6 : Online Classif ication Technical requirements Python environment Defining classification Identifying use cases of classification Use case 1 – email spam classification Use case 2 – face detection in phone camera Use case 3 – online marketing ad selection Overview of classification algorithms in River Classification algorithm 1 – LogisticRegression Classification algorithm 2 – Perceptron Classification algorithm 3 – AdaptiveRandomForestClassifier Classification algorithm 4 – ALMAClassifier Classification algorithm 5 – PAClassifier Evaluating benchmark results Summary Further reading
Page
14
Chapter 7 : Online Regression Technical requirements Python environment Defining regression Use cases of regression Use case 1 – Forecasting Use case 2 – Predicting the number of faulty products in manufacturing Overview of regression algorithms in River Regression algorithm 1 – LinearRegression Regression algorithm 2 – HoeffdingAdaptiveTreeRegressor Regression algorithm 3 – SGTRegressor Regression algorithm 4 – SRPRegressor Summary Further reading
Page
15
Chapter 8 : Reinforcement Learning Technical requirements Python environment Defining reinforcement learning Comparing online and offline reinforcement learning A more detailed overview of feedback loops in reinforcement learning The main steps of a reinforcement learning model Making the decisions Updating the decision rules Exploring Q-learning The goal of Q-learning Parameters of the Q-learning algorithm Deep Q-learning Using reinforcement learning for streaming data Use cases of reinforcement learning Use case one – trading system Use case two – social network ranking system Use case three – a self-driving car Use case four – chatbots Use case five – learning games Implementing reinforcement learning in Python Summary Further reading
Page
16
Part 3: Advanced Concepts and Best Practices around Streaming Data
Page
17
Chapter 9 : Drif t and Drif t Detection Technical requirements Python environment Defining drift Three types of drift Introducing model explicability Measuring drift Measuring data drift Measuring concept drift Measuring drift in Python A basic intuitive approach to measuring drift Measuring drift with robust tools Counteracting drift Offline learning with retraining strategies against drift Online learning against drift Summary Further reading
Page
18
Chapter 10 : Feature Transformation and Scaling Technical requirements Python environment Challenges of data preparation with streaming data Scaling data for streaming Introducing scaling Adapting scaling to a streaming context Transforming features in a streaming context Introducing PCA Mathematical definition of PCA Regular PCA in Python Incremental PCA for streaming Summary Further reading
Page
19
Chapter 11 : Catastrophic Forgett ing Technical requirements Python environment Introducing catastrophic forgetting Catastrophic forgetting in online models Detecting catastrophic forgetting Using Python to detect catastrophic forgetting Model explicability versus catastrophic forgetting Explaining models using linear coefficients Explaining models using dendrograms Explaining models using variable importance Summary Further reading
Page
20
Chapter 12 : Conclusion and Best Practices Going further Summary Other Books You May Enjoy
The above is a preview of the first 20 pages. Register to read the complete e-book.
AI Reading Assistant
Whole-book reading guide from stratified index samples; jump to passages in the text
AI guide
【One-Line Pitch】
A hands-on guide to building real-time machine learning systems with Python, showing how to move beyond batch thinking toward continuously learning models. Best suited for data scientists and ML practitioners who already know standard scikit-learn workflows and now need to handle live, unbounded data streams.
【Book Arc】
- **Opening (~0%–15%)**: Frames the core problem — most analytics are still done in batch even when real-time value is possible — and contrasts batch versus streaming approaches, including the advantages of real-time data-generating processes.
- **Early (~15%–30%)**: Builds the architectural foundation: turning analytics into functions, sending HTTP requests, and standing up a minimal real-time inference API (e.g., via AWS API Gateway and Lambda), while flagging the software-engineering concerns a data scientist should not own alone.
- **Early–Middle (~30%–45%)**: Covers streaming descriptive statistics (rolling windows, mode, standard deviation), real-time visualization with Plotly Dash, and alerting/monitoring systems, including statistical process control for more advanced alerts.
- **Middle (~45%–55%)**: Introduces online machine learning as distinct from offline learning — models that update sequentially rather than in a single training pass — demonstrated with River on the iris dataset and learning-curve inspection.
- **Late (~55%–85%)**: Moves into industrial use cases such as online anomaly detection and a PCA alternative that works on real-time data, plus the challenges that arise when streaming and how to mitigate them.
- **Ending (~85%–100%)**: Consolidates best practices for generating real-time insights and building an API for real-time machine learning inference, aiming to leave the reader confident deploying streaming models.
【Key Takeaways】
- **Streaming is an architectural choice, not just a modeling one** (Early): the book treats real-time analytics as a pipeline — function, API, deployment — before touching model internals.
- **Batch and streaming coexist** (Early): many use cases still run in batch simply because those methods are more familiar; the book helps you judge when streaming adds genuine value.
- **Descriptive statistics need rethinking on streams** (Early–Middle): rolling windows, mode, and standard deviation are recomputed over sliding windows rather than fixed datasets.
- **Alerting evolves from static rules to statistical process control** (Middle): simple threshold alerts on single values give way to logic based on means, medians, and process stability.
- **Online learning updates models sequentially** (Middle): unlike offline train-deploy-retrain cycles, online models keep learning from each new observation, which suits drifting data.
- **River is the primary toolkit** (Middle): the book uses River for online models (e.g., logistic regression on iris) and inspects learning curves to judge stability.
- **Real-time visualization is achievable with lightweight tools** (Middle): Plotly Dash with interval callbacks provides near-real-time dashboards without heavy infrastructure.
- **Deployment and inference APIs close the loop** (Late): the book walks through exposing models for real-time inference, connecting analytics to production consumers.
【Reading Tips】
- Deep-read the early architecture chapters if you have never built an API; skim them if you already deploy services regularly.
- Treat the River code examples as the core — type them out rather than copying, since the sequential `learn_one` pattern is easy to misapply.
- Pay attention to the "challenges and mitigations" material in the later chapters; that is where the practical judgment lives.
- Use the GitHub repository alongside the text; the book explicitly points to notebooks and a real-time visualization script there.
- If you are strong on statistics, skim the descriptive-statistics refresher and focus on how windows change the computation.
【Coverage Limits】
The excerpts cover the book's framing, architecture, descriptive statistics, visualization, alerting, and the introduction to online learning with River, but do not detail the specific anomaly detection methods, the PCA alternative implementation, or the full best-practices chapter. Chapter titles and exact percentages beyond the markers shown are not verifiable from the excerpts.
Passage locations
Excerpt 1
Develop a PCA alternative that can work on real-time data Explore best practices for handling streaming data that you absolutely need to remember Develop an...
View in text
Excerpt 2
this book, you can download the code in the repository and execute it using your preferred Python editor. If you are not yet familiar with Python environment...
View in text
Excerpt 3
loper-guide/kinesis-examples- capturing-page-scrolling.html measure, as you'll never have enough time and money to measure every individual in the population...
View in text
Excerpt 4
d use it for predicting your input stream. You can probably track the performance metrics of your model, and when the performance starts to change, you can d...
View in text
Recommended for You
{{#thumbnailUrl}}
{{/thumbnailUrl}}
{{^thumbnailUrl}}
{{/thumbnailUrl}}
Loading recommended books...
Failed to load, please try again later
Tip the Site
Scan the WeChat Pay or Alipay code to tip. No login required.
WeChat Pay
Alipay