Multithreaded JavaScript Concurrency Beyond the Event Loop (Thomas Hunter II, Bryan English) (Z-Library)

📄 File Format: PDF

💾 File Size: 5.8 MB

📖 Read Online ⬇️ Download

Registered users can read the full content for free

Register as a Gaohf Library member to read the complete e-book online for free and enjoy a better reading experience.

📄 Page 1

Thomas Hunter II & Bryan English Multithreaded JavaScript Concurrency Beyond the Event Loop

📄 Page 2

(This page has no text content)

📄 Page 3

Thomas Hunter II and Bryan English Multithreaded JavaScript Concurrency Beyond the Event Loop Boston Farnham Sebastopol TokyoBeijing

📄 Page 4

978-1-098-10443-6 [LSI] Multithreaded JavaScript by Thomas Hunter II and Bryan English Copyright © 2022 Thomas Hunter II and Bryan English. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com. Acquisitions Editor: Amanda Quinn Development Editor: Corbin Collins Production Editor: Daniel Elfanbaum Copyeditor: Tom Sullivan Proofreader: nSight, Inc. Indexer: nSight, Inc. Interior Designer: David Futato Cover Designer: Karen Montgomery Illustrator: Kate Dullea October 2021: First Edition Revision History for the First Edition 2021-09-22: First Release See http://oreilly.com/catalog/errata.csp?isbn=9781098104436 for release details. The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Multithreaded JavaScript, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc. The views expressed in this work are those of the authors, and do not represent the publisher’s views. While the publisher and the authors have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the authors disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights.

📄 Page 5

This book is dedicated to Katelyn and Renée.

📄 Page 6

(This page has no text content)

📄 Page 7

Table of Contents Foreword. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi 1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 What Are Threads? 3 Concurrency Versus Parallelism 5 Single-Threaded JavaScript 6 Hidden Threads 9 Threads in C: Get Rich with Happycoin 10 With Only the Main Thread 11 With Four Worker Threads 13 2. Browsers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 Dedicated Workers 20 Dedicated Worker Hello World 20 Advanced Dedicated Worker Usage 23 Shared Workers 25 Shared Worker Hello World 27 Advanced Shared Worker Usage 32 Service Workers 33 Service Worker Hello World 35 Advanced Service Worker Concepts 40 Message Passing Abstractions 43 The RPC Pattern 43 The Command Dispatcher Pattern 45 Putting It All Together 47 v

📄 Page 8

3. Node.js. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 Before We Had Threads 54 The worker_threads Module 56 workerData 57 MessagePort 58 Happycoin: Revisited 60 With Only the Main Thread 60 With Four Worker Threads 63 Worker Pools with Piscina 65 A Pool Full of Happycoins 69 4. Shared Memory. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 Intro to Shared Memory 74 Shared Memory in the Browser 74 Shared Memory in Node.js 77 SharedArrayBuffer and TypedArrays 79 Atomic Methods for Data Manipulation 84 Atomics.add() 85 Atomics.and() 85 Atomics.compareExchange() 86 Atomics.exchange() 86 Atomics.isLockFree() 86 Atomics.load() 86 Atomics.or() 87 Atomics.store() 87 Atomics.sub() 87 Atomics.xor() 87 Atomicity Concerns 88 Data Serialization 91 Booleans 91 Strings 93 Objects 94 5. Advanced Shared Memory. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 Atomic Methods for Coordination 97 Atomics.wait() 98 Atomics.notify() 99 Atomics.waitAsync() 100 Timing and Nondeterminism 100 Example of Nondeterminism 100 Detecting Thread Preparedness 104 Example Application: Conway’s Game of Life 106 vi | Table of Contents

📄 Page 9

Single-Threaded Game of Life 107 Multithreaded Game of Life 112 Atomics and Events 118 6. Multithreaded Patterns. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 Thread Pool 121 Pool Size 122 Dispatch Strategies 123 Example Implementation 125 Mutex: A Basic Lock 131 Streaming Data with Ring Buffers 137 Actor Model 144 Pattern Nuances 145 Relating to JavaScript 146 Example Implementation 146 7. WebAssembly. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 Your First WebAssembly 155 Atomic Operations in WebAssembly 157 Compiling C Programs to WebAssembly with Emscripten 159 Other WebAssembly Compilers 160 AssemblyScript 161 Happycoin in AssemblyScript 163 8. Analysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 When Not to Use 169 Low Memory Constraints 170 Low Core Count 172 Containers Versus Threads 175 When to Use 176 Summary of Caveats 181 Appendix. Structured Clone Algorithm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187 Table of Contents | vii

📄 Page 10

(This page has no text content)

📄 Page 11

Foreword The book you’re holding now is a fun one. It’s a JavaScript book that opens with examples written in C, talks about multithreading with an explicitly single-threaded programming language, provides great examples of how and when to intentionally block the event loop after experts have been telling you for years to never do so, and closes with an excellent list of reasons and caveats about why you might not actually want to use the mechanisms the book describes. More importantly, it’s a book that I would consider essential reading for any JavaScript developer no matter where your code is expected to be deployed and run. When I’ve worked with companies to help them build more efficient and more per‐ formant Node.js and JavaScript applications, I’ve often had to step back and take the time first to discuss many of the common misconceptions developers have about the programming language. For instance, I once had an engineer with a long history in Java and .NET development argue that creating a new promise in JavaScript was a lot like creating a new thread in Java (it’s not), and that promises allow JavaScript to run in parallel (they don’t). In a separate conversation someone had created a Node.js application that was spawning over a thousand simultaneous worker threads and wasn’t sure why they weren’t seeing an expected improvement in performance while testing on a machine that had only eight logical CPU cores. The lesson from these conversations is clear: multithreading, concurrency, and parallelism are still very unfamiliar and difficult topics for a very large percentage of JavaScript developers. Dealing with these misconceptions is what led directly to me (working with my col‐ league and fellow Node.js Technical Steering Committee member, Matteo Collina) developing the Broken Promises workshop in which we would lay out the founda‐ tions of asynchronous programming in JavaScript—teaching engineering teams how to reason more effectively about the order in which their code would execute and the timing of various events. It also led directly to the development of the Piscina open source project (with fellow Node.js core contributor Anna Henningsen), which pro‐ vides a best-practice implementation of a worker pool model on top of Node.js worker threads. But those only help with part of the challenge. ix

📄 Page 12

In this book, Bryan and Thomas expertly lay out the foundations of multithreaded development in general, and deftly illustrate how the various JavaScript runtimes like web browsers and Node.js enable parallel computing with a programming language that includes no built-in mechanisms to enable it. Because the responsibility for pro‐ viding multithreading support has fallen on the runtimes, and because there are so many differences between those runtimes, browsers and platforms like Node.js imple‐ ment multithreading in different ways. Although they share similar APIs, a worker thread in Node.js is really not the same thing as a web worker in a web browser. Sup‐ port for shared workers, web workers, and service workers is nearly universal across browsers, and worker threads have been in Node.js for several years now, but they are all still a relatively new concept for JavaScript developers. No matter where your Java‐ Script runs, this book will provide important insight and information. Most impor‐ tantly, however, the authors take the time to explain exactly why you should care at all about multithreading in your JavaScript applications. — James Snell, Node.js Technical Steering Committee Member x | Foreword

📄 Page 13

Preface Bryan and I (Thomas) first met during my interview at the San Francisco branch for DeNA, a Japanese mobile game development company. Apparently most of the upper management was going to say no, but after the two of us hung out at a Node.js meetup later that night, Bryan went and convinced them to give me an offer. While at DeNA, Bryan and I worked on writing reusable Node.js modules so that game teams could build out their game servers, combining components as appropri‐ ate to suit the needs of their game. Performance was something we were always meas‐ uring, and mentoring game teams on performance was a part of the job; our servers were continuously scrutinized by developers in an industry that traditionally relied upon C++. The two of us would work together in other capacities as well. Another such role was at a small security startup named Intrinsic where we focused on hardening Node.js apps at such a complete and granular level that I doubt the world will ever see another product like it. Performance tuning was a huge concern for that product as well since customers didn’t want to take a hit to their throughput. We spent many hours run‐ ning benchmarks, poring over flamegraphs, and digging through internal Node.js code. Had the worker threads module been available in all the versions of Node.js that our customers demanded, I have no doubt we would have incorporated it into the product. We’ve also worked together in nonemployment capacities as well. NodeSchool SF is one such example wherein we both volunteered to teach others how to use JavaScript and create Node.js programs. We have also spoken at many of the same conferences and meetups. Both of your authors have a passion for JavaScript and Node.js, and for teaching them to others and eliminating misconceptions. When we realized there was such an extreme lack of documentation about building multithreaded JavaScript applications, we knew what we had to do. This book was born from our desire to not only educate others about the capabilities of JavaScript, but also to help prove that platforms like xi

📄 Page 14

Node.js are just as capable as any other when it comes to building performant serv‐ ices that utilize the available hardware. Target Audience The ideal reader of this book is an engineer who has been writing JavaScript for a few years, and who doesn’t necessarily have experience with writing multithreaded appli‐ cations or even experience with more traditionally multithreaded languages like C++ or Java. We do include some example C application code, as a sort of multithreaded lingua franca, but it’s not something that the reader is expected to be familiar with or even understand. If you do have experience with such languages, that’s great, and this book will help you understand the JavaScript equivalent to the functionality provided by whatever language you may be familiar with. On the other hand, if you’ve only written code using JavaScript, then this book is also for you. We include information across multi‐ ple layers of learning; this includes both low-level API references, high-level patterns, and plenty of technical tangents in between to help fill in any gaps. Goals Perhaps the most exuberant goal of this book is to bring knowledge to the community that it’s possible to build multithreaded applications using JavaScript. Traditionally, JavaScript code was constrained to a single core, and indeed there are many Twitter threads and forum posts describing the language as such. With a title like Multithrea‐ ded JavaScript, we hope to completely dispel the notion that JavaScript applications are confined to a single core. At a more concrete level, the goal is to teach you, the reader, several aspects about writing multithreaded JavaScript applications. By the time you’re done reading this book you’ll understand the various web worker APIs provided in browsers, their strengths and weaknesses, and when to use which. As far as Node.js goes, you’ll understand the worker threads module and how its APIs compare to those in the browser. The book focuses on two approaches to building multithreaded applications: one using message passing and the other using shared memory. By reading this book you’ll understand the APIs used to implement each, when you might want to use one approach or the other, and in which situations they can be combined—and you’ll even get your hands dirty with some high-level patterns built upon these approaches. xii | Preface

📄 Page 15

Conventions Used in This Book The following typographical conventions are used in this book: Italic Indicates new terms, URLs, email addresses, filenames, and file extensions. Constant width Used for program listings, as well as within paragraphs to refer to program ele‐ ments such as variable or function names, databases, data types, environment variables, statements, and keywords. Constant width bold Shows commands or other text that should be typed literally by the user. Constant width italic Shows text that should be replaced with user-supplied values or by values deter‐ mined by context. This element signifies a tip or suggestion. This element signifies a general note. This element indicates a warning or caution. Using Code Examples Supplemental material (code examples, exercises, etc.) is available for download at https://github.com/MultithreadedJSBook/code-samples. If you have a technical question or a problem using the code examples, please send email to bookquestions@oreilly.com. Preface | xiii

📄 Page 16

This book is here to help you get your job done. In general, if example code is offered with this book, you may use it in your programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require permission. We appreciate, but generally do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: “Multithreaded JavaScript by Thomas Hunter II and Bryan English (O’Reilly). Copyright 2022 Thomas Hunter II and Bryan English, 978-1-098-10443-6.” If you feel your use of code examples falls outside fair use or the permission given above, feel free to contact us at permissions@oreilly.com. O’Reilly Online Learning For more than 40 years, O’Reilly Media has provided technol‐ ogy and business training, knowledge, and insight to help companies succeed. Our unique network of experts and innovators share their knowledge and expertise through books, articles, and our online learning platform. O’Reilly’s online learning platform gives you on-demand access to live training courses, in-depth learning paths, interactive coding environments, and a vast collection of text and video from O’Reilly and 200+ other publishers. For more information, visit http://oreilly.com. How to Contact Us Please address comments and questions concerning this book to the publisher: O’Reilly Media, Inc. 1005 Gravenstein Highway North Sebastopol, CA 95472 800-998-9938 (in the United States or Canada) 707-829-0515 (international or local) 707-829-0104 (fax) We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at https://oreil.ly/multithreaded-js. xiv | Preface

📄 Page 17

Email bookquestions@oreilly.com to comment or ask technical questions about this book. For news and information about our books and courses, visit http://oreilly.com. Find us on Facebook: http://facebook.com/oreilly. Follow us on Twitter: http://twitter.com/oreillymedia. Watch us on YouTube: http://www.youtube.com/oreillymedia. Acknowledgments This book was made possible thanks to the detailed technical reviews provided by the following people: Anna Henningsen (@addaleax) Currently part of the MongoDB Developer Tools team in Germany, Anna has been one of the most active contributors to Node.js core over the last five years, and participated significantly in implementing worker threads for the platform. She is fueled by a passion for Node.js and its community. Shu-yu Guo (@_shu) Shu works on JavaScript implementation and standardization. He is a TC39 dele‐ gate, one of the editors of the ECMAScript specification, and the author of the memory model. He currently works on the V8 engine at Google, leading Java‐ Script language feature implementation and standards. Previously, he has worked at Mozilla and Bloomberg. Fernando Larrañaga (@xabadu) Fernando is an engineer and open source contributor who has been leading Java‐ Script and Node.js communities for several years both in South America and in the United States. He’s currently a senior software engineer at Square and an organizer of NodeSchool SF, and with previous tenures at other major tech com‐ panies—such as Twilio and Groupon—he has been developing enterprise-level Node.js and scaling web applications used by millions of users since 2014. Preface | xv

📄 Page 18

(This page has no text content)

📄 Page 19

CHAPTER 1 Introduction Computers used to be much simpler. That’s not to say they were easy to use or write code for, but conceptually there was a lot less to work with. PCs in the 1980s typically had a single 8-bit CPU core and not a whole lot of memory. You typically could only run a single program at one time. What we think of these days as operating systems would not even be running at the same time as the program the user was interacting with. Eventually, people wanted to run more than one program at once, and multitasking was born. This allowed operating systems to run several programs at the same time by switching execution between them. Programs could decide when it would be an appropriate time to let another program run by yielding execution to the operating system. This approach is called cooperative multitasking. In a cooperative multitasking environment, when a program fails to yield execution for any reason, no other program can continue executing. This interruption of other programs is not desirable, so eventually operating systems moved toward preemptive multitasking. In this model, the operating system would determine which program would run on the CPU at which time, using its own notion of scheduling, rather than relying on the programs themselves to be the sole deciders of when to switch execu‐ tion. To this day, almost every operating system uses this approach, even on multi-core systems, because we generally have more programs running than we have CPU cores. Running multiple tasks at once is extremely useful for both programmers and users. Before threads, a single program (that is, a single process) could not have multiple tasks running at the same time. Instead, programmers wishing to perform tasks con‐ currently would either have to split up the task into smaller chunks and schedule them inside the process or run separate tasks in separate processes and have them communicate with each other. 1

📄 Page 20

Even today, in some high-level languages the appropriate way to run multiple tasks at once is to run additional processes. In some languages, like Ruby and Python, there’s a global interpreter lock (GIL), meaning only one thread can be executing at a given time. While this makes memory management far more practical, it makes multithrea‐ ded programming not as attractive to programmers, and instead multiple processes are employed. Until fairly recently, JavaScript was a language where the only multitasking mecha‐ nisms available were splitting tasks up and scheduling their pieces for later execution, and in the case of Node.js, running additional processes. We’d typically break code up into asynchronous units using callbacks or promises. A typical chunk of code written in this manner might look something like Example 1-1, breaking up the operations by callbacks or await. Example 1-1. A typical chunk of asynchronous JavaScript code, using two different patterns readFile(filename, (data) => { doSomethingWithData(data, (modifiedData) => { writeFile(modifiedData, () => { console.log('done'); }); }); }); // or const data = await readFile(filename); const modifiedData = await doSomethingWithData(data); await writeFile(filename); console.log('done'); Today, in all major JavaScript environments, we have access to threads, and unlike Ruby and Python, we don’t have a GIL making them effectively useless for perform‐ ing CPU-intensive tasks. Instead, other trade-offs are made, like not sharing Java‐ Script objects across threads (at least not directly). Still, threads are useful to JavaScript developers for cordoning off CPU-intensive tasks. In the browser, there are also special-purpose threads that have feature sets available to them that are different from the main thread. The details of how we can do this are the topics of later chap‐ ters, but to give you an idea, spawning a new thread and handling a message in a browser can be as simple as Example 1-2. 2 | Chapter 1: Introduction