Refactoring to Rust (Lily Mara, Joel Holmes) (Z-Library)

M A N N I N G Lily Mara Joel Holmes

Refactoring to Rust

ii

Refactoring to Rust LILY MARA JOEL HOLMES M A N N I N G SHELTER ISLAND

For online information and ordering of this and other Manning books, please visit www.manning.com. The publisher offers discounts on this book when ordered in quantity. For more information, please contact Special Sales Department Manning Publications Co. 20 Baldwin Road PO Box 761 Shelter Island, NY 11964 Email: orders@manning.com ©2025 by Manning Publications Co. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by means electronic, mechanical, photocopying, or otherwise, without prior written permission of the publisher. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in the book, and Manning Publications was aware of a trademark claim, the designations have been printed in initial caps or all caps. Recognizing the importance of preserving what has been written, it is Manning’s policy to have the books we publish printed on acid-free paper, and we exert our best efforts to that end. Recognizing also our responsibility to conserve the resources of our planet, Manning books are printed on paper that is at least 15 percent recycled and processed without the use of elemental chlorine. The author and publisher have made every effort to ensure that the information in this book was correct at press time. The author and publisher do not assume and hereby disclaim any liability to any party for any loss, damage, or disruption caused by errors or omissions, whether such errors or omissions result from negligence, accident, or any other cause, or from any usage of the information herein. Manning Publications Co. Development editor: Elesha Hyde 20 Baldwin Road Technical development editor: Mark Elston PO Box 761 Review editor: Kishor Rit Shelter Island, NY 11964 Production editor: Aleksandar Dragosavljević Copy editor: Alisa Larson Proofreader: Katie Tennant Technical proofreader: Jerry Kuch Typesetter and cover designer: Marija Tudor ISBN 9781617299018 Printed in the United States of America

To anyone who thought that they couldn't —Lily Mara To my Grandma Jo and my Aunt Alisha who inspired my love of reading and technology —Joel Holmes

brief contents 1 ■ Why refactor to Rust 1 2 ■ An overview of Rust 15 3 ■ Introduction to C FFI and unsafe Rust 60 4 ■ Advanced FFI 93 5 ■ Structuring Rust libraries 138 6 ■ Integrating with dynamic languages 164 7 ■ Testing your Rust integrations 187 8 ■ Asynchronous Python with Rust 212 9 ■ WebAssembly for refactoring JavaScript 229 10 ■ WebAssembly interface for refactoring 249 vi

contents preface xi acknowledgments xii about this book xiv about the authors xvii about the cover illustration xviii 1 Why refactor to Rust 1 1.1 What is refactoring? 2 1.2 What is Rust? 4 1.3 Why Rust? 4 1.4 Should you refactor to Rust? 5 Performance 5 ■ Memory safety 7 ■ Maintainability 7 1.5 When not to refactor to Rust 9 1.6 How does it work? 9 1.7 What will you learn in this book? 11 Calling Rust functions directly from your program 11 Communicating with a Rust service over the network 12 1.8 Who is this book for? 13 1.9 What tools do you need to get started? 13vii

CONTENTSviii2 An overview of Rust 15 2.1 Ownership and borrowing 16 2.2 Memory management in other languages 19 2.3 Lifetimes 23 References and borrowing 26 ■ Controlling mutability 28 References and lifetimes 30 2.4 Rust’s string types 33 Mutable strings 34 2.5 Enums and error handling 37 Enums 37 ■ Error handling with enums 41 ■ The unit type 43 ■ Error types 45 ■ Transforming errors 49 Panicking with errors 53 3 Introduction to C FFI and unsafe Rust 60 3.1 Unsafe Rust 61 Raw pointers 61 3.2 C Foreign Function Interface 64 Including a crate 68 ■ Creating a dynamic library with Rust 70 ■ Solving arithmetic expressions in Rust 77 The Display trait 87 4 Advanced FFI 93 4.1 Downloading the NGINX source code 94 4.2 Creating the NGINX module 94 4.3 Linking C to Rust 98 Build scripts 100 ■ bindgen 103 4.4 Reading the NGINX request 109 Lifetime annotations 115 ■ Lifetime annotations in our NGINX plugin 120 4.5 Using our calculator library 124 4.6 Writing the HTTP response 128 5 Structuring Rust libraries 138 5.1 Modules 138 Who cares? 142 ■ Multiple files 143 5.2 Paths 146 Relative vs. absolute pathspaths 147 ■ Path aliases 155 5.3 Upward visibility 159

CONTENTS ix6 Integrating with dynamic languages 164 6.1 Data processing in Python 164 6.2 Planning the move 165 6.3 JSON Parsing 166 6.4 Writing a Python extension module in Rust 171 6.5 Benchmarking in Rust 176 6.6 Optimized builds 184 7 Testing your Rust integrations 187 7.1 Writing tests with Rust 187 Documentation tests 193 ■ Adding tests to existing code 198 7.2 Testing Rust code using Python 202 Monkey patching 206 8 Asynchronous Python with Rust 212 8.1 Generating a Mandelbrot set in Python 213 8.2 Scaling 215 8.3 Asyncio 218 8.4 Threading 220 8.5 Global Interpreter Lock 223 8.6 PyO3 224 9 WebAssembly for refactoring JavaScript 229 9.1 What is WebAssembly? 230 9.2 Moving from JavaScript to Rust 231 9.3 Rust in the browser 232 Requesting data 232 ■ Compiling to Wasm 235 Loading Wasm in the browser 237 9.4 Creating a React component 238 9.5 Web components entirely in Rust 241 9.6 Refactoring JavaScript revisited 247 10 WebAssembly interface for refactoring 249 10.1 WASI universal runtime 252 10.2 From the browser to the machine 255 10.3 Wasm library 261

CONTENTSx10.4 Consuming Wasm 262 10.5 More Wasm 267 10.6 Wasm memory 270 10.7 Just the beginning 275 index 277

preface Throughout our software careers, we’ve had the opportunity to participate in several refactoring projects. The narrative is often the same: products need to scale, but time is limited. This situation leads to extensive development efforts over months, filled with discussions about patterns and languages. Refactoring with Java and Go involved significant challenges, including constant file moving, package exports, system wrappers, and outright rewrites of existing sys- tems. The paths to success were rarely clearly defined. This book aims to provide you with many of these patterns, using a language designed for breaking down and rewrit- ing existing systems. Refactoring to Rust demonstrates how Rust can seamlessly integrate into your ecosystem, delivering scaling benefits from day one due to the nature of the language. Rust brings advantages, such as type safety and memory safety, along with perfor- mance gains attributed to these properties. In this book, you will learn how Rust can enhance nearly any project. Positioned to replace existing languages like C and C++, Rust stands out for its robust toolchain and memory safety features. We will also explore how Rust can interact with languages like Python, revealing performance improvements when building libraries and modules that work across both languages. Additionally, we’ll discover unexpected uses for Rust, such as in web browsers and as a universal runtime. Overall, this book aims not only to showcase the power of Rust but also to equip you with the skills to refactor large systems with confidence.xi

acknowledgments I thank my partners for encouraging me to get this book over the finish line after I let it languish for so long. To my mother, whose love of reading kept my nose in books for years and who edited some early drafts, thank you. I thank my parents and grandparents for always encouraging me to pursue a tech- nical education and making that possible. Thank you, the trans community, everywhere that you are. There is no world without us. To my Uncle Conrad, thank you for helping me develop my curiosity in the way things work. To everyone at Manning, thank you. This book would not exist without the dedica- tion of many editors, technical editors, graphics editors, marketers, proofreaders, and more. I thank Andy Waldron for the chance to write my first book. A special thank you goes to the editors of this book, Elesha Hyde and Susan Ethridge. Your guidance led this book out of the vagaries of my thought into the real world. To OneSignal, thank you for giving me the time and freedom to write this book. I thank the teachers and professors who nurtured an appreciation for the written word within me—Kristen Schumacher, Paul Hebert, and the late Jean Lutz. Thank you, Norm Krumpe, for feeding my technical curiosity, and Dr. Paris Franz for encouraging me to do something as big as writing a book. Thank you, Jan Pascual, for being the guinea pig behind the technical communi- cation skills that underpin this writing, and for your encouragement throughout the process.xii

ACKNOWLEDGMENTS xiii Writing a book is a herculean effort, and I also thank everyone who was an early reviewer or a MEAP customer who left feedback in the online forums for the book. —LILY MARA First, I would like to thank my wife and partner, Chelsea, who encourages me to pur- sue my dreams of writing and learning. I also thank my two sons, Eli and Abel, who are an endless source of inspiration. My dedication of this book is to two important women in my life. The first is my grandmother, who instilled my love of books and reading, and second is my Aunt Ali- sha, who influenced my love of computers at an early age. This book could not have been written without the immense support of Manning’s publishing team. Thank you, Andy Waldron, for this opportunity to write a book on such an exciting subject. As a reviewer of many Manning books myself, I especially appreciate all those who provided feedback in the book reviews: Alain Couniot, Alfred Thompson, Amit Lamba, Ariel Otilibili, Chris Kardell, Christopher Villanueva, Clifford Thurber, Dan Sheikh, Daniel Tomás Lares, Diego Alonso, Federico Kircheis, Foster Haines, Gabor Laszlo Hajba, Gilles Iachelini, Havard Wall, Jahred Love, James Blachly, John Kasiewicz, Jon Riddle, Jonathan Reeves, Julien Castelain, Kent R. Spillner, Krzysztof Kamyczek, Maciej Przepióra, Marcus Geselle, Matthew Sarmiento, Max Sadrieh, Michal Rutka, Mohsen Mostafa Jokar, Ramon Snir, Rani Sharim, Richard Randall, Sal- vador Navarrete Garcia, Sam Van Overmeire, Sambasiva Andaluri, Seung-jin Kim, Seyi Ogunyemi, Tim McNamara, Troi Eisler, Walt Stoneburner, William E. Wheeler, and Yerkebulan Tulibergenov. And I appreciate those who purchased this book early via MEAP and provided feedback and support. I am very grateful for all the help, guidance, and patience that Elesha Hyde pro- vided. She had the difficult task of shepherding this book throughout the process with immense grace. Her patience and support are greatly appreciated. Thanks also go to Regrow.ag, who gave me the freedom and encouragement to write this book; my friend Cody, who has been there for me since elementary school; my high school English teachers, who encouraged my writing and helped me establish my voice; and Otto, for always being there to listen and never to judge. —JOEL HOLMES

about this book Martin Fowler’s renowned book Refactoring emphasizes the primary goal of refactor- ing: to enhance the design of existing code. Readers familiar with the book will recog- nize its method of presenting various code segments, followed by improved alternatives that enhance readability, efficiency, or simplicity. While the strategies have evolved in the second edition, the core message remains unchanged: functional code can always be improved. Refactoring to Rust outlines strategies for transitioning from one programming lan- guage to another while preserving the external behavior of the code. How is this achieved? As we will examine, Rust is designed to gradually replace other languages by integrating and decomposing existing code—much like the process of rusting iron— and substituting it with Rust code. Initially focused on replacing C++, the project has expanded to include JavaScript and Python. Who should read this book This book is focused on developers who specialize in other languages, such as C, C++, Python, and JavaScript, but want to learn Rust. While this book does not give you an in-depth view of the language, it does provide practical examples and use cases to change your code to Rust. No formal understanding of Rust is needed, although it is helpful. How this book is organized: A roadmap In line with Fowler’s approach, we will present challenges in one language and demonstrate how to refactor these complexities in Rust. The goal is to maintain the underlying functionality of the application while using Rust’s speed and safety to enhance the overall system.xiv

ABOUT THIS BOOK xv Our exploration begins with an introduction to the Rust language, discussing its mechanics and comparing it to languages like C, C++, and Python. This information is framed within the context of refactoring, emphasizing how we can systematically improve our systems instead of allowing them to devolve into unmanageable code. We will also delve into Rust’s advanced features, such as variable lifetime and ownership, which are crucial to mastering the language. The first major focus will be on C, the foundational language for many others. In chapter 3, we will examine Rust’s ability to create both safe and unsafe code, explore wrapping dangerous code in Rust, and utilize debugging tools. This foundation will prepare us for chapter 4, where we will integrate Rust into an existing C codebase, manipulate memory, and add new functionality to an NGINX server. After our initial integration into another system, in chapter 5, we will consider Rust as a library tool. Creating packages compatible with other projects is an effective way to refactor applications, provided that these libraries offer enhanced functionality. We will also explore benchmarking and performance metrics to justify the transition from older languages to Rust. In chapters 6, 7, and 8, we will demonstrate how these pack- ages can be used to refactor Python code, either by executing Python within Rust or by embedding Rust into Python. The final two chapters will challenge us with advanced applications of Rust. Chap- ter 9 will focus on compiling Rust to run in web browsers using a new format called Wasm. Chapter 10 will use this technology to build a universal runtime, providing a flexible (yet complex) method for refactoring or interacting with existing code. The chapters are not required to be read in order, and if you are already familiar with Rust, you can probably skip the first two chapters unless you want a refresher. If you are eager to jump into a particular language, chapters 3 and 4 focus on integrat- ing with C and C++, and chapters 6 through 8 focus on Python, while chapter 9 focuses on JavaScript. Chapter 10 can also be read on its own and offers a different way to refactor by changing the environment in which an application runs rather than changing the code itself. Refactoring is more art than science. Both Martin Fowler’s book and ours offer patterns to follow; it will be your responsibility to apply these techniques effectively. About the code The code covered in this book mostly focuses on Rust, but within the context of other languages. The basics of Rust are covered at the beginning, and then integration with C, Python, and JavaScript occurs throughout the remainder of the book. These lan- guages are not taught but are expected to be known by the reader if they are refactor- ing code in that language. There are no limitations on hardware or the software used. Nothing in the text is specific to a particular operating system or requires any special setup other than an

ABOUT THIS BOOKxviinstallation of Rust. Additional libraries and tools are mentioned in the chapters, but the text is dedicated to this setup and is not required by the reader to do beforehand. In addition, Rust is a growing language, and therefore, the syntax and libraries may shift over time. We have taken care to select stable libraries in our examples to accommodate this as much as possible. The book contains many examples of source code in numbered listings and in line with normal text. In both cases, source code is formatted in a fixed-width font like this to separate it from ordinary text. In many cases, the original source code has been reformatted; we’ve added line breaks and reworked indentation to accommodate the available page space in the book. In some cases, even this is not enough, and listings include line-continuation markers (➥). Additionally, comments in the source code have often been removed from the listings when the code is described in the text. Code annotations accompany many of the listings, highlighting important concepts. You can get executable snippets of code from the liveBook (online) version of this book at https://livebook.manning.com/book/refactoring-to-rust. The complete code for the examples in the book is available for download from the Manning website at https://www.manning.com/books/refactoring-to-rust, and from GitHub at https:// github.com/lily-mara/refactoring-to-rust. liveBook discussion forum Purchase of Refactoring to Rust includes free access to liveBook, Manning’s online read- ing platform. Using liveBook’s exclusive discussion features, you can attach comments to the book globally or to specific sections or paragraphs. It’s a snap to make notes for yourself, ask and answer technical questions, and receive help from the author and other users. To access the forum, go to https://livebook.manning.com/book/ refactoring-to-rust/discussion. Manning’s commitment to our readers is to provide a venue where a meaningful dialogue between individual readers and between readers and the authors can take place. It is not a commitment to any specific amount of participation on the part of the authors, whose contribution to the forum remains voluntary (and unpaid). We suggest you try asking the authors some challenging questions lest their interests stray! The forum and the archives of previous discussions will be accessible from the pub- lisher’s website as long as the book is in print.

about the authors LILY MARA is a software developer based in San Francisco, Califor- nia. She speaks domestically and internationally about Rust soft- ware development. She has been writing Rust since 2015 and uses it professionally for writing high-performance, scalable systems. She is currently writing software at Discord. JOEL HOLMES is a software developer who has been focused on building cloud-native applications. He has worked at several start- ups to architect, design, and develop new products and services to help those companies develop and grow. Along the way, he was able to help establish tools and processes that helped development and increase quality. He lives in Pittsburgh with his family and cur- rently works building cloud applications at Regrow.ag. . xvii

about the cover illustration The figure on the cover of Refactoring to Rust is “Piemontoise d’Asti,” or “A Woman from City of Asti in Piedmont,” taken from a collection by Jacques Grasset de Saint- Sauveur, published in 1788. This illustration is finely drawn and colored by hand. In those days, it was easy to identify where people lived and what their trade or sta- tion in life was just by their dress. Manning celebrates the inventiveness and initiative of the computer business with book covers based on the rich diversity of regional cul- ture centuries ago, brought back to life by pictures from collections such as this one. xviii

Why refactor to RustIf you have ever heard of the Rust programming language, you may have heard of software companies rewriting their code in Rust from a slower, interpreted lan- guage. A few of these companies have published blog posts lauding the perfor- mance benefits of Rust over their previous systems, and they tell a very tidy story: other languages are slow, and Rust is fast. Therefore, rewrite your code in Rust, and your systems will be fast. While it may be tempting to think that we can all just rewrite our code when some- thing better comes along, we all know the reality that software does not exist in a bub- ble of infinite resources. Performance improvements and technical debt payments need to be balanced with feature development, user requests, and the million other This chapter covers  Why you may want to refactor an application  Why Rust is a good choice for refactoring  When it is and is not appropriate to start a refactoring project  A high-level overview of methods you can use to refactor your code into Rust1