Share E-Book

Rust for Machine Learning (Marcos Silveira)(Z-Library)

Author Marcos Silveira

rust
Language English

This book invites you to move beyond high-level libraries and abstractions to explore the fundamentals of machine learning through hands-on coding. Along the way, you'll gain a practical understanding of how AI systems work while learning Rust, one of today's most powerful and expressive programming languages. Author Marcos Silveira offers a real-world journey into the intersection of math, systems programming, and machine learning. Written for developers who want to expand their skills and data practitioners curious about what's happening under the hood, this book equips you with the tools to build, debug, and adapt intelligent systems with confidence.

Format PDF
Size 2.7 MB
12
Views
0
Downloads
0.00
Total Donations
(First 20 pages)

Registered users can read the full content for free

Register as a Gaohf Library member to read the complete e-book online for free and enjoy a better reading experience.

Page 1
(This page has no text content)
Page 2
Rust for Machine Learning Learn AI Fundamentals and Build a Neural Network from Scratch With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles. Marcos Silveira
Page 3
Rust for Machine Learning by Marcos Silveira Copyright © 2027 Marcos Silveira. All rights reserved. Published by O’Reilly Media, Inc. , 141 Stony Circle, Suite 195, Santa Rosa, CA 95401. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (https://oreilly.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com. Acquisitions Editor: Louise Corrigan Development Editor: Melissa Potter Production Editor: Elizabeth Faerm Copyeditor: TO COME Proofreader: TO COME Indexer: TO COME Cover Designer: Susan Brown Cover Illustrator: Monica Kamsvaag Interior Designer: David Futato Interior Illustrator: Kate Dullea August 2027: First Edition Revision History for the Early Release
Page 4
2026-07-23: First Release See https://oreilly.com/catalog/errata.csp?isbn=9798341672789 for release details. The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Rust for Machine Learning, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc. The views expressed in this work are those of the author and do not represent the publisher’s views. While the publisher and the author have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights. 979-8-341-67274-1 [LSI]
Page 5
Brief Table of Contents (Not Yet Final) Preface (unavailable) Introduction: Programs that Learn (available) Chapter 1: The First Neuron (available) Chapter 2: The Search for Structure (available) Chapter 3: A Framework for Learning (unavailable) Chapter 4: Building the Toolkit (unavailable) Chapter 5: Representations by Hand (unavailable) Chapter 6: From Hard Decisions to Probabilities (unavailable) Chapter 7: Optimization (unavailable) Chapter 8: Our First Neural Network (unavailable) Chapter 9: The “Deep” in Deep Learning (unavailable) Chapter 10: Targeting Visual Learning with Convolutional Neural Networks (unavailable) Chapter 11: Practical Considerations of Running Neural Networks (unavailable) Chapter 12: Scientific Computing (unavailable) Chapter 13: Intro to Unsupervised Learning (unavailable) Chapter 14: Intro to Reinforcement Learning (unavailable) Chapter 15: A Glimpse into Transformers (unavailable)
Page 6
Introduction: Programs that Learn A NOTE FOR EARLY RELEASE READERS With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles. This will be the introduction of the final book. Please note that the GitHub repo will be made active later on. If you’d like to be actively involved in reviewing and commenting on this draft, please reach out to the editor at mpotter@oreilly.com. [M]achines of this character can behave in a very complicated manner when the number of units is large. —Alan Turing When I first started working with machine learning, more than ten years ago, I was fascinated by the idea of systems that can learn from data. At that time, we were building a demand forecasting service for a client, using historical contract and sales data to predict demand for different components in the following year. To help us achieve our goal we brought a data scientist in the team pairing with developers to deliver a testable, production-ready system. He soon started experimenting with different solutions for the problem, one of them being linear regression, a machine learning technique that predicts numerical outputs based on the data. I wanted to understand the theory behind what we were building, so a colleague and I spent our nights in a 24- hour bakery working through Andrew Ng’s Stanford Machine Learning course. At that time, it was still taught in Octave, a programming language designed for scientific use. By then, I was already an experienced programmer. I had written many pieces of code that explicitly defined step-by-step computations like getting
Page 7
the total value from a list of invoices or implementing a multi-thread multi- step data transformation. I was used to coding very complex business logic that required detailed modeling, testing, integration with other modules, and interacting with other systems. Programming was (and still is) a powerful tool to solve all kinds of problems. But then machine learning entered the picture. What is Machine Learning? Machine learning refers to programs that learn to solve a specific task just by looking at data, rather than by following explicit coding rules. This data can be of all sorts: text, images, videos, voice recordings, spreadsheets, PDFs, tabular data, completely unstructured documents. The machine learning program then receives the examples of data as input and runs some internal process to learn the rules to solve that specific problem. And once the program learns the patterns on this data, it can be used to solve similar tasks on different data. It can classify the unseen data, search and extract information, or recognize patterns in new samples. But how? I remember my admiration for the idea behind a relatively small computer program being able to achieve so much. Consider a few of the tasks modern machine learning can perform: translating text, generating speech and video, writing emails, creating images, generating code. Not too long ago, these were tasks that were only possible for humans. More importantly, these are tasks that are impossible to solve by explicitly programming. How can one realistically write rules for an open-ended text translator or a prompt-based image generation tool? What kind of programs make such rich and complex behavior possible? This is what I am going to unravel in this book. I want to go deep into the mechanics of machine learning to understand how these programs learn the way they learn.
Page 8
Understanding Machine Learning by Building We will certainly code and configure machine learning programs, but we will go beyond that. Richard Feynman had this sentence written on his blackboard at the time of his death: “What I cannot build, I do not understand.” I interpret this phrase as a statement about true understanding: something is only truly known when we are able to build it from first principles. In other words, it is through building something from scratch that we encounter the small details that lead to real understanding. This is true for mathematics and physics as it is true for machine learning. And so we will build. My approach is to begin with the foundational pieces of machine learning and make steady progress toward more complex solutions. As we go, we will build intuition in three distinct dimensions of machine learning: representation, learning, and inference. Representation is about understanding the task we want to solve and choosing the right class of models capable of expressing a solution. Learning uses data to fit a model to the task at hand. Finally, we have inference, which takes the learned model and applies it to new data. As we advance through the book, we will gradually introduce more expressive models and more advanced learning techniques. Through concrete examples, we will observe how these models expand the range of tasks that can be solved. We will conclude by building a neural network, one of the most flexible and expressive classes of models in use today. By the end of the book, you will have a solid, working understanding of how neural networks actually operate under the hood. In the real world, when building machine learning systems, we are often facing a plethora of challenges: it starts with the data that is frequently noisy, incomplete, or unavailable; also the learning algorithm may seem to fail as the parameters are hard to fine-tune or the weights are vanishing to zero; when the tasks are more complex, the computing resources required for optimization increase beyond practical limits; choosing the right model
Page 9
class is an art as it relies on intuition of what can work or not; and many times, debugging the learning process is cumbersome. This book takes a pragmatic approach. Rather than discussing theory and implementation in isolation, we will tackle these practical difficulties directly. You will learn how to reason about model behavior, diagnose training failures, improve convergence, and decide which approach fits a given problem. These challenges are added to the fact that many data scientists and AI engineers got so used to high-level tooling. Modern libraries make it possible to assemble sophisticated systems quickly, but they often hide the complexity of the inner workings. As a result, debugging, testing, and improving models become harder, and many AI projects end up failing. Don’t get me wrong, I don’t think high-level libraries are not useful. They speed up experimentation, enable progress at scale, and bring production grade performance. But I do believe people using these tools should know what happens beneath the surface. Only then one is equipped to design robust models, diagnose failures, and build systems that deliver lasting value in real-world settings. This is exactly what the book aims to provide. Why Rust For this book I have chosen Rust. The main reason is that Rust helps us to make the mechanics of machine learning explicit, rather than hiding it behind abstractions. Two aspects of the language are particularly relevant: 1. Rust encourages first-principles implementation. Even though Rust has libraries for data manipulation, linear algebra, and machine learning, its ecosystem is much smaller and less known when compared to Python’s or R’s (the two dominant programming languages in the field.) We take advantage of this. Instead of relying on established libraries such as NumPy, Pandas, or SciPy to perform the heavy lifting, we will implement the core components
Page 10
ourselves. This constraint is deliberate as it keeps the focus on understanding. 2. Rust exposes what other languages and libraries usually hide. Concepts like strong typing, ownership, immutability, generics… force us into thinking about implementation details that are usually implicit elsewhere. We must explicitly design our model internal state for mutability, take care of data ownership when sending through the different parts of the program, and eventually build our own type system to use in our linear algebra library implementation. Other than that, Rust is just fun. It has been for the 10th year the most admired programming language in the Stackoverflow yearly research1. Also, its design for memory management, speed, and productivity is a huge differential. As we go through the book, you will learn more about Rust’s special features, especially its take on mutability, the idea of ownership as a first-class design element, and the many things that make Rust fast. The Field of Machine Learning Before we move into our first machine learning program, let me briefly comment on the content strategy for this book. Machine learning is a big field of study, and it can be categorized in many ways. Figure I-1 shows the two categorization strategies we are using in this book.
Page 11
(This page has no text content)
Page 12
Figure I-1. Subdivisions on machine learning First, it is often divided according to the different ways of learning: Supervised learning The program learns by comparing its predictions to a known solution provided by another author (labeled data), typically by a human. Unsupervised learning The program learns based on the structure of the data itself, with no information about solutions coming from the outside. Self-supervised learning The program builds its own learning target from the data itself. This set of techniques was popularized recently by its use in training large language models. Reinforcement learning The program learns from trial and error, receiving rewards or penalties on its performance, and optimizing a specific metric. More recently, due to the advance of Large Language Models, another classification of the field has gained highlight, that between discriminative models and generative models: Discriminative models These models focus on learning the differences between classes or conditional relationships, such as distinguishing between dogs and cats, predicting whether a patient has a disease, or estimating house prices.
Page 13
Generative models These models aim to capture the underlying structure of the data itself. Once trained, they can generate new samples, such as images of dogs, text in French, or a voice sample of Obama. How does the book fit in this schema? The book’s end goal is to have a convolutional deep neural network, which is a specific type of neural network powerful for classifying images. That specific use case fits under supervised learning on the first classification schema, and under discriminative learning on the second. But that is not the whole story. Neural networks are not limited to supervised classification. They serve as the fundamental building blocks of generative models and are also used for self-supervised tasks and reinforcement learning. By building a neural network from scratch, you will have the foundations for being able to apply on all those learning paradigms. (The next paragraphs will be updated as the book progresses.) But building neural networks is only our end goal. To get there, we will go through many other components typically seen in a machine learning course. Chapters 1-7 lay out the foundation for neural networks, including constructing different types of neurons (ch. 1, 2, 5, 6), tackling learning theory with a pragmatic standpoint (ch. 3), and the problem of optimization (ch. 7). Chapters 8-10 focus on neural networks, from introduction (ch. 8), to deep neural networks (ch. 9) and convolutional neural networks (ch. 10). In chapter 11 we will go through different aspects of building, testing, deploying, and evolving models. Chapter 12 takes on the challenging topic of scientific computing, in our case, how to optimize code for performance. Chapters 13-15 use neural networks to touch on different machine learning paradigms: first we build some unsupervised learning algorithms (cf. 13), then markov decision processes (MDPs) and Q-learning networks on ch. 14
Page 14
and finally on chapter 15. we build a micro transformer and a word embedding. Chapter 4 is an exception, as it is focused on Rust and on refactoring our code to give it a more modular structure we will need for the following chapters. (consider removing the table from here so save some space in the chapter) Table I-0 summarizes what we will be building and provides a bit more of details on the specifics that are going to appear in each chapter.
Page 15
Chapter Model Dataset Rust Othe 1. Learning from Data Perceptron AND, OR, XOR Foundations of the language 2. Data classifiers Marginal classifiers Data generating functions Ownership 3. Framework of learning KNNs Valid testin valid 4. Types in Rust Types and modules 5. Logistic regression and neurons Logistic regression Activ funct cross valid 6. How about non-linear data? SVM, Kernels XOR, other 7. Optimization Grad desce stoch gradi desce
Page 16
Chapter Model Dataset Rust Othe 8. Our first neural network Feed forward neural network Back propa 9. The deep in Deep Learning Deep neural networks Featu const 10. Visual learning Convolutional neural networks Images 11. More practical considerations 12. Scientific computing 13. Touching unsupervised learning 14. Intro to Reinforcement learning 15. A glimpse into transformers
Page 17
Audience, Resources, and Prerequisites This book is written for readers with some programming experience who want to learn machine learning while also gaining a solid understanding of Rust. You may be a data scientist, data engineer, machine learning engineer, or a software developer who has some practice in the field, perhaps using Python and its ecosystem, but feels that what happens under the hood remains unclear. If that is the case, then this book is for you. We will focus on the details and nuances that make machine learning systems actually work. You may also be completely new to practical machine learning and looking for a structured way in. In that case, this book is also for you. By the time you finish it, you will be able to onboard quickly onto machine learning frameworks in Python, or any other language, with a clear understanding of the underlying principles. If you are concerned about having little knowledge of Rust, don’t be. A secondary goal of this book is to teach the language itself. We will start from the fundamentals and gradually move toward more advanced features of the language. You will learn Rust the most effective way possible: by building real stuff and seeing the results of your work. If, on the other hand, you already know Rust and are worried that this book may feel repetitive, this is unlikely to be the case. Even though in the beginning we lay out the basics of Rust, we do that by writing meaningful machine learning programs. We then move quickly to more advanced and idiomatic Rust, which should remain engaging even for experienced Rust developers. Ultimately, this book is for readers who want to move beyond using machine learning as a black box. Whether your primary motivation is to master Rust, to understand the mechanics of learning algorithms, or to have a toolbox of practical techniques to improve results, the goal is the same: to build systems from first principles and understand every component along
Page 18
the way. By the end, you will not only know how to use machine learning tools—you will understand why they work. Mathematics and the Companion Website Implementing machine learning programs requires familiarity with certain mathematical concepts. When writing about machine learning, there is a constant trade-off: do I want to keep the build flow going, or do I want to interrupt the flow to explain deeply the mathematical foundations. In this book, we introduce only the minimum amount of math necessary to support a clear understanding of the techniques and systems we build at each stage. This choice is deliberate: it will allow us to maintain a sense of progress and focus on the central goal of the book, which is building a neural network from first principles. This approach favors conceptual clarity and continuity over full formal rigor. The material presented here is entirely sufficient to understand, implement, and reason about the systems we build throughout the book. For readers who enjoy exploring the mathematics in greater depth, a companion website (www.math4rustnn.com ) offers optional supplementary notes with more formal derivations and extended discussions. These resources are meant to enrich the journey, not to complete it. I still expect some high school mathematics concepts to be clear. Here is a short discussion on what is expected from the reader: (I will be completing this section as the book progresses) Linear Algebra This is the first mathematical topic that we will encounter in the book. In machine learning data is often represented as vectors. We will be using bold face caps letters to represent vectors. So X = [ 1, 2, 3, . . . , ] is a vector with dimensions.
Page 19
Vectors can be added so, for = 2, we have X + Y = [ 1, 2] + [ 1, 2] = [ 1 + 1, 2 + 2]. Vectors can also be scaled by a number, so for a real number , we have X = [ 1, 2] = [ 1, 2] Transpose, dot operations and norms Probability Optimization Calculus Supporting code Our goal is to implement the core components of machine learning and neural networks ourselves. This forces us to think carefully about what belongs to the heart of a learning system and what can be delegated to supporting tools. As a guiding principle, we will keep all logic that is central to machine learning, such as data manipulation and linear algebra, within our own codebase. In contrast, we will rely on external libraries only for functionality that is peripheral to the learning process, such as randomness generation and plotting. In practice, this means that we will write our own data transformation utilities and linear algebra routines, while using established libraries or supporting code for tasks that are far removed from the conceptual core of machine learning. All code accompanying this book is available in the GitHub repository: https://github.com/bymarkone/rustnn Preparing the environment
Page 20
To follow along with the examples in this book, you will need a working Rust installation. The recommended way to install Rust is through rustup, the official toolchain installer and version manager. It handles downloading the compiler (rustc), the package manager (cargo), and keeping your environment up to date. Installation instructions for all major operating systems are available at https://www.rust-lang.org . Once installed, you can verify everything is working by running rustc --version and cargo --version in your terminal. Rust projects are managed with Cargo, which is both a build system and dependency manager. To create a new project, run cargo new project_name, which generates a ready-to-use directory structure. You can then build and run your program with cargo run. If you are using an IDE or code editor, most modern environments provide excellent Rust support through the rust-analyzer extension. Visual Studio Code is a common choice, but any editor with Language Server Protocol (LSP) support will work well. Once Rust and Cargo are properly installed, no additional configuration is required to begin building the systems developed in this book. We are now ready to start! Our first step is tiny, we will build a classifier that uses the Perceptron algorithm to learn from data. But we will already touch on many elements that will serve us through the book. 1 https://survey.stackoverflow.co/2025/technology#admired-and-desired
The above is a preview of the first 20 pages. Register to read the complete e-book.

Support Author

0.00
Total Amount (¥)
0
Donation Count
Please enter an amount Minimum ¥1

You will be redirected to Alipay to complete payment, then return here.

Recommended for You

Loading recommended books...
Failed to load, please try again later
Back to List