Zero to Production in Rust (Luca Palmieri) (Z-Library)

📄 File Format: PDF

💾 File Size: 4.4 MB

📖 Read Online ⬇️ Download

Registered users can read the full content for free

Register as a Gaohf Library member to read the complete e-book online for free and enjoy a better reading experience.

📄 Page 1

(This page has no text content)

📄 Page 2

Contents Foreword xiii Preface xv What Is This Book About . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv Cloud-native applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv Working in a team . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvi Who Is This Book For . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii 1 Getting Started 1 1.1 Installing The Rust Toolchain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1.1 Compilation Targets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1.2 Release Channels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.1.3 What Toolchains DoWeNeed? . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2 Project Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.3 IDEs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.3.1 Rust-analyzer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.3.2 IntelliJ Rust . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.3.3 What Should I Use? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.4 Inner Development Loop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.4.1 Faster Linking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.4.2 cargo-watch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.5 Continuous Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.5.1 CI Steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.5.2 Ready-to-go CI Pipelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2 Building An Email Newsletter 11 2.1 Our Driving Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.1.1 Problem-based Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.2 What Should Our Newsletter Do? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.2.1 Capturing Requirements: User Stories . . . . . . . . . . . . . . . . . . . . . . 12 2.3 Working In Iterations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.3.1 Coming Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.4 Checking Your Progress . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 3 Sign Up A New Subscriber 15 3.1 Our Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 i

📄 Page 3

ii CONTENTS 3.2 Choosing AWeb Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 3.3 Our First Endpoint: A Basic Health Check . . . . . . . . . . . . . . . . . . . . . . . . . . 16 3.3.1 Wiring Up actix-web . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 3.3.2 Anatomy Of An actix-webApplication . . . . . . . . . . . . . . . . . . . . . . 18 3.3.3 Implementing The Health Check Handler . . . . . . . . . . . . . . . . . . . . 23 3.4 Our First Integration Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 3.4.1 HowDo You Test An Endpoint? . . . . . . . . . . . . . . . . . . . . . . . . . . 26 3.4.2 Where Should I Put My Tests? . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 3.4.3 Changing Our Project Structure For Easier Testing . . . . . . . . . . . . . . . . 28 3.5 Implementing Our First Integration Test . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 3.5.1 Polishing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 3.6 Refocus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 3.7 WorkingWith HTML Forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 3.7.1 Refining Our Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 3.7.2 Capturing Our Requirements As Tests . . . . . . . . . . . . . . . . . . . . . . 41 3.7.3 Parsing FormData From A POSTRequest . . . . . . . . . . . . . . . . . . . . . 43 3.8 Storing Data: Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 3.8.1 Choosing A Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 3.8.2 Choosing A Database Crate . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 3.8.3 Integration TestingWith Side-effects . . . . . . . . . . . . . . . . . . . . . . . . 55 3.8.4 Database Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 3.8.5 Writing Our First Query . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 3.9 Persisting ANew Subscriber . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 3.9.1 Application State In actix-web . . . . . . . . . . . . . . . . . . . . . . . . . . 71 3.9.2 actix-webWorkers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 3.9.3 The Data Extractor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 3.9.4 The INSERTQuery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 3.10 Updating Our Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 3.10.1 Test Isolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 3.11 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 4 Telemetry 89 4.1 Unknown Unknowns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 4.2 Observability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 4.3 Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 4.3.1 The logCrate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 4.3.2 actix-web’s LoggerMiddleware . . . . . . . . . . . . . . . . . . . . . . . . . . 92 4.3.3 The Facade Pattern . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 4.4 Instrumenting POST /subscriptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 4.4.1 Interactions With External Systems . . . . . . . . . . . . . . . . . . . . . . . . 96 4.4.2 Think Like A User . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 4.4.3 Logs Must Be Easy To Correlate . . . . . . . . . . . . . . . . . . . . . . . . . . 99 4.5 Structured Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 4.5.1 The tracingCrate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 4.5.2 Migrating From logTo tracing . . . . . . . . . . . . . . . . . . . . . . . . . . 102

📄 Page 4

CONTENTS iii 4.5.3 tracing’s Span . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 4.5.4 Instrumenting Futures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 4.5.5 tracing’s Subscriber . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 4.5.6 tracing-subscriber . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 4.5.7 tracing-bunyan-formatter . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 4.5.8 tracing-log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 4.5.9 Removing Unused Dependencies . . . . . . . . . . . . . . . . . . . . . . . . . 113 4.5.10 Cleaning Up Initialisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 4.5.11 Logs For Integration Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 4.5.12 Cleaning Up Instrumentation Code - tracing::instrument . . . . . . . . . . 121 4.5.13 Protect Your Secrets - secrecy . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 4.5.14 Request Id . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 4.5.15 Leveraging The tracing Ecosystem . . . . . . . . . . . . . . . . . . . . . . . . 130 4.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 5 Going Live 133 5.1 WeMust Talk About Deployments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 5.2 Choosing Our Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134 5.2.1 Virtualisation: Docker . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134 5.2.2 Hosting: DigitalOcean . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 5.3 A Dockerfile For Our Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 5.3.1 Dockerfiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 5.3.2 Build Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136 5.3.3 Sqlx OfflineMode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 5.3.4 Running An Image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 5.3.5 Networking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 5.3.6 Hierarchical Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 5.3.7 Database Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146 5.3.8 Optimising Our Docker Image . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 5.4 Deploy To DigitalOcean Apps Platform . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152 5.4.1 Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 5.4.2 App Specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 5.4.3 How To Inject Secrets Using Environment Variables . . . . . . . . . . . . . . . 156 5.4.4 Connecting To Digital Ocean’s Postgres Instance . . . . . . . . . . . . . . . . . 158 5.4.5 Environment Variables In The App Spec . . . . . . . . . . . . . . . . . . . . . 162 5.4.6 One Last Push . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 6 Reject Invalid Subscribers #1 165 6.1 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 6.1.1 Domain Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 6.1.2 Security Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 6.2 First Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167 6.3 Validation Is A Leaky Cauldron . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170 6.4 Type-Driven Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171 6.5 Ownership Meets Invariants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174

📄 Page 5

iv CONTENTS 6.5.1 AsRef . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178 6.6 Panics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180 6.7 Error As Values - Result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182 6.7.1 Converting parse To Return Result . . . . . . . . . . . . . . . . . . . . . . . 183 6.8 Insightful Assertion Errors: claims . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185 6.9 Unit Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186 6.10 Handling A Result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188 6.10.1 match . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189 6.10.2 The ? Operator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189 6.10.3 400 Bad Request . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190 6.11 The Email Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191 6.12 The SubscriberEmail Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192 6.12.1 Breaking The Domain Sub-Module . . . . . . . . . . . . . . . . . . . . . . . . 192 6.12.2 Skeleton Of ANew Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 6.13 Property-based Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196 6.13.1 How To Generate Random Test Data With fake . . . . . . . . . . . . . . . . . 196 6.13.2 quickcheck Vs proptest . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 6.13.3 Getting StartedWith quickcheck . . . . . . . . . . . . . . . . . . . . . . . . . 197 6.13.4 Implementing The Arbitrary Trait . . . . . . . . . . . . . . . . . . . . . . . . 198 6.14 Payload Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201 6.14.1 RefactoringWith TryFrom . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205 6.15 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208 7 Reject Invalid Subscribers #2 209 7.1 Confirmation Emails . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209 7.1.1 Subscriber Consent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209 7.1.2 The Confirmation User Journey . . . . . . . . . . . . . . . . . . . . . . . . . . 210 7.1.3 The Implementation Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . 210 7.2 EmailClient, Our Email Delivery Component . . . . . . . . . . . . . . . . . . . . . . . . 211 7.2.1 How To Send An Email . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211 7.2.2 How ToWrite A REST Client Using reqwest . . . . . . . . . . . . . . . . . . 214 7.2.3 How To Test A REST Client . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222 7.2.4 First Sketch Of EmailClient::send_email . . . . . . . . . . . . . . . . . . . . 227 7.2.5 Tightening Our Happy Path Test . . . . . . . . . . . . . . . . . . . . . . . . . 235 7.2.6 DealingWith Failures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243 7.3 Skeleton And Principles For AMaintainable Test Suite . . . . . . . . . . . . . . . . . . . . 253 7.3.1 Why DoWeWrite Tests? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253 7.3.2 Why Don’t WeWrite Tests? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254 7.3.3 Test Code Is Still Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254 7.3.4 Our Test Suite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255 7.3.5 Test Discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256 7.3.6 One Test File, One Crate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256 7.3.7 Sharing Test Helpers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257 7.3.8 Sharing Startup Logic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260 7.3.9 Build An API Client . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269

📄 Page 6

CONTENTS v 7.3.10 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273 7.4 Refocus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273 7.5 Zero Downtime Deployments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274 7.5.1 Reliability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274 7.5.2 Deployment Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275 7.6 Database Migrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278 7.6.1 State Is Kept Outside The Application . . . . . . . . . . . . . . . . . . . . . . . 278 7.6.2 Deployments AndMigrations . . . . . . . . . . . . . . . . . . . . . . . . . . . 278 7.6.3 Multi-stepMigrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279 7.6.4 ANewMandatory Column . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279 7.6.5 ANew Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281 7.7 Sending A Confirmation Email . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282 7.7.1 A Static Email . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282 7.7.2 A Static Confirmation Link . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287 7.7.3 Pending Confirmation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291 7.7.4 Skeleton of GET /subscriptions/confirm . . . . . . . . . . . . . . . . . . . . 294 7.7.5 Connecting The Dots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297 7.7.6 Subscription Tokens . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307 7.8 Database Transactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315 7.8.1 All Or Nothing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315 7.8.2 Transactions In Postgres . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315 7.8.3 Transactions In Sqlx . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316 7.9 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320 8 Error Handling 323 8.1 What Is The Purpose Of Errors? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323 8.1.1 Internal Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324 8.1.2 Errors At The Edge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326 8.1.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328 8.2 Error Reporting For Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329 8.2.1 Keeping Track Of The Error Root Cause . . . . . . . . . . . . . . . . . . . . . 331 8.2.2 The Error Trait . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337 8.3 Errors For Control Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341 8.3.1 Layering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341 8.3.2 Modelling Errors as Enums . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342 8.3.3 The Error Type Is Not Enough . . . . . . . . . . . . . . . . . . . . . . . . . . . 344 8.3.4 Removing The Boilerplate With thiserror . . . . . . . . . . . . . . . . . . . . 348 8.4 Avoid “Ball Of Mud” Error Enums . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350 8.4.1 Using anyhowAs Opaque Error Type . . . . . . . . . . . . . . . . . . . . . . . 355 8.4.2 anyhowOr thiserror? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357 8.5 Who Should Log Errors? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 358 8.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359 9 Naive Newsletter Delivery 361 9.1 User Stories Are Not Set In Stone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361

📄 Page 7

vi CONTENTS 9.2 Do Not SpamUnconfirmed Subscribers . . . . . . . . . . . . . . . . . . . . . . . . . . . 362 9.2.1 Set Up State Using The Public API . . . . . . . . . . . . . . . . . . . . . . . . 364 9.2.2 ScopedMocks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364 9.2.3 Green Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365 9.3 All Confirmed Subscribers Receive New Issues . . . . . . . . . . . . . . . . . . . . . . . . 366 9.3.1 Composing Test Helpers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 366 9.4 Implementation Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368 9.5 Body Schema . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369 9.5.1 Test Invalid Inputs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370 9.6 Fetch Confirmed Subscribers List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372 9.7 Send Newsletter Emails . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374 9.7.1 context Vs with_context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375 9.8 Validation Of Stored Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376 9.8.1 Responsibility Boundaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 380 9.8.2 Follow The Compiler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382 9.8.3 Remove Some Boilerplate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383 9.9 Limitations Of The Naive Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385 9.10 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 386 10 Securing Our API 387 10.1 Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387 10.1.1 Drawbacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 388 10.1.2 Multi-factor Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . 388 10.2 Password-based Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 388 10.2.1 Basic Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 388 10.2.2 Password Verification - Naive Approach . . . . . . . . . . . . . . . . . . . . . . 394 10.2.3 Password Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 398 10.2.4 Do Not Block The Async Executor . . . . . . . . . . . . . . . . . . . . . . . . 415 10.2.5 User Enumeration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 422 10.3 Is it safe? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427 10.3.1 Transport Layer Security (TLS) . . . . . . . . . . . . . . . . . . . . . . . . . . 427 10.3.2 Password Reset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427 10.3.3 Interaction Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427 10.3.4 Machine ToMachine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427 10.3.5 Person Via Browser . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428 10.3.6 Machine to machine, on behalf of a person . . . . . . . . . . . . . . . . . . . . 429 10.4 Interlude: Next Steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 429 10.5 Login Forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 429 10.5.1 Serving HTML Pages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 429 10.6 Login . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 432 10.6.1 HTML Forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433 10.6.2 Redirect On Success . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 436 10.6.3 Processing FormData . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 436 10.6.4 Contextual Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445 10.7 Sessions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 478

📄 Page 8

CONTENTS vii 10.7.1 Session-based Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . 478 10.7.2 Session Store . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 478 10.7.3 Choosing A Session Store . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 479 10.7.4 actix-session . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 480 10.7.5 Admin Dashboard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 483 10.8 Seed Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 495 10.8.1 Database Migration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 495 10.8.2 Password Reset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 497 10.9 Refactoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514 10.9.1 How ToWrite An actix-webMiddleware . . . . . . . . . . . . . . . . . . . . . 515 10.10 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 522 11 Fault-tolerant Workflows 525 11.1 POST /admin/newsletters - A Refresher . . . . . . . . . . . . . . . . . . . . . . . . . . . 525 11.2 Our Goal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 527 11.3 Failure Modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 527 11.3.1 Invalid Inputs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 527 11.3.2 Network I/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 528 11.3.3 Application Crashes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 529 11.3.4 Author Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 529 11.4 Idempotency: An Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 529 11.4.1 Idempotency In Action: Payments . . . . . . . . . . . . . . . . . . . . . . . . . 530 11.4.2 Idempotency Keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 530 11.4.3 Concurrent Requests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 531 11.5 Requirements As Tests #1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 532 11.6 Implementation Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 533 11.6.1 Stateful Idempotency: Save And Replay . . . . . . . . . . . . . . . . . . . . . . 533 11.6.2 Stateless Idempotency: Deterministic Key Generation . . . . . . . . . . . . . . 533 11.6.3 Time Is a Tricky Beast . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 534 11.6.4 Making A Choice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 534 11.7 Idempotency Store . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 534 11.7.1 Which Database ShouldWe Use? . . . . . . . . . . . . . . . . . . . . . . . . . . 534 11.7.2 Schema . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535 11.8 Save And Replay . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 536 11.8.1 Read Idempotency Key . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 536 11.8.2 Retrieve Saved Responses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 541 11.8.3 Save Responses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 544 11.9 Concurrent Requests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 551 11.9.1 Requirements As Tests #2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 552 11.9.2 Synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 553 11.10 DealingWith Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 560 11.10.1 Distributed Transactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 562 11.10.2 Backward Recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 563 11.10.3 Forward Recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 563 11.10.4 Asynchronous Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 564

📄 Page 9

viii CONTENTS 11.11 Epilogue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 581

📄 Page 10

(This page has no text content)

📄 Page 11

(This page has no text content)

📄 Page 12

A Federica, il mio porto sicuro.

📄 Page 13

(This page has no text content)

📄 Page 14

Foreword When you read these lines, Rust has achieved its biggest goal: make an offer to programmers to write their production systems in a different language. By the end of the book, it is still your choice to follow that path, but you have all you need to consider the offer. I’ve been part of the growth process of two widely different languages: Ruby and Rust - by programming them, but also by running events, being part of their project management and running business around them. Through that, I had the privilege of being in touch with many of the creators of those languages and consider some of them friends. Rust has been my one chance in life to see and help a language grow from the experimental stage to adoption in the industry. I’ll let you in on a secret I learned along theway: programming languages are not adopted because of a feature checklist. It’s a complex interplay between good technology, the ability to speak about it and finding enough people willing to take long bets. When I write these lines, over 5000 people have contributed to the Rust project, often for free, in their spare time - because they believe in that bet. But you don’t have to contribute to the compiler or be recorded in a git log to contribute to Rust. Luca’s book is such a contribution: it gives newcomers a perspective on Rust and promotes the good work of those many people. Rust was never intended to be a research platform - it was always meant as a programming language solving real, tangible issues in large codebases. It is no surprise that it comes out of an organization that maintains a very large and complex codebase -Mozilla, creators of Firefox. When I joinedRust, it was just ambition - but the ambitionwas to industrialize research tomake the software of tomorrowbetter. With all of its theoretical concepts, linear typing, region based memory management, the programming language was always meant for everyone. This reflects in its lingo: Rust uses accessible names like “Ownership” and “Borrowing” for the concepts I just mentioned. Rust is an industry language, through and through. And that reflects in its proponents: I’ve known Luca for years as a community member who knows a ton about Rust. But his deeper interest lies in convincing people that Rust is worth a try by addressing their needs. The title and structure of this book reflects one of the core values of Rust: to find its worth in writing production software that is solid and works. Rust shows its strength in the care and knowledge that went into it to write stable software productively. Such an experience is best found with a guide and Luca is one of the best guides you can find around Rust. Rust doesn’t solve all of your problems, but it has made an effort to eliminate whole categories of mistakes. There’s the view out there that safety features in languages are there because of the incompetence of pro- grammers. I don’t subscribe to this view. Emily Dunham, captured it well in her RustConf 2017 keynote: “safe code allows you to take better risks”. Much of the magic of the Rust community lies in this positive view of its users: whether you are a newcomer or an experienced developer, we trust your experience and your decision-making. In this book, Luca offers a lot of new knowledge that can be applied even outside of xiii

📄 Page 15

xiv CONTENTS Rust, well explained in the context of daily software praxis. I wish you a great time reading, learning and contemplating. Florian Gilcher, Management Director of Ferrous Systems and Co-Founder of the Rust Foundation

📄 Page 16

Preface What Is This Book About The world of backend development is vast. The context you operate into has a huge impact on the optimal tools and practices to tackle the problem you are working on. For example, trunk-based developmentworks extremelywell towrite software that is continuously deployed in a Cloud environment. The very same approach might fit poorly the business model and the challenges faced by a team that sells software that is hosted and run on-premise by their customers - they aremore likely to benefit from aGitflow approach. If you are working alone, you can just push straight to main. There are few absolutes in the field of software development and I feel it’s beneficial to clarify your point of view when evaluating the pros and cons of any technique or approach. Zero To Production will focus on the challenges of writing Cloud-native applications in a team of four or five engineers with different levels of experience and proficiency. Cloud-native applications Defining what Cloud-native application means is, by itself, a topic for a whole new book1. Instead of pre- scribing what Cloud-native applications should look like, we can lay down what we expect them to do. Paraphrasing Cornelia Davis, we expect Cloud-native applications: • To achieve high-availability while running in fault-prone environments; • To allow us to continuously release new versions with zero downtime; • To handle dynamic workloads (e.g. request volumes). These requirements have a deep impact on the viable solution space for the architecture of our software. High availability implies that our application should be able to serve requests with no downtime even if one or more of our machines suddenly starts failing (a common occurrence in a Cloud environment2). This 1Like the excellent Cloud-native patterns by Cornelia Davis! 2For example, many companies run their software on AWS Spot Instances to reduce their infrastructure bills. The price of Spot xv

📄 Page 17

xvi CONTENTS forces our application to be distributed - there should be multiple instances of it running on multiple ma- chines. The same is true if we want to be able to handle dynamic workloads - we should be able tomeasure if our system is under load and throw more compute at the problem by spinning up new instances of the applica- tion. This also requires our infrastructure to be elastic to avoid overprovisioning and its associated costs. Running a replicated application influences our approach to data persistence - we will avoid using the local filesystem as our primary storage solution, relying instead on databases for our persistence needs. Zero To Productionwill thus extensively cover topics thatmight seem tangential to pure backend application development. But Cloud-native software is all about rainbows and DevOps, therefore we will be spending plenty of time on topics traditionally associated with the craft of operating systems. We will cover how to instrument your Rust application to collect logs, traces and metrics to be able to observe our system. We will cover how to set up and evolve your database schema via migrations. We will cover all the material required to use Rust to tackle both day one and day two concerns of a Cloud- native API. Working in a team The impact of those three requirements goes beyond the technical characteristics of our system: it influences how we build our software. To be able to quickly release a new version of our application to our users we need to be sure that our applic- ation works. If you are working on a solo project you can rely on your thorough understanding of the whole system: you wrote it, it might be small enough to fit entirely in your head at any point in time.3 If you areworking in a teamon a commercial project, youwill be very oftenworking on code thatwas neither written or reviewed by you. The original authors might not be around anymore. You will end up being paralysed by fear every time you are about to introduce changes if you are relying on your comprehensive understanding of what the code does to prevent it from breaking. You want automated tests. Running on every commit. On every branch. Keeping main healthy. You want to leverage the type system to make undesirable states difficult or impossible to represent. You want to use every tool at your disposal to empower each member of the team to evolve that piece of software. To contribute fully to the development process even if they might not be as experienced as you or equally familiar with the codebase or the technologies you are using. instances is the result of a continuous auction and it can be substantially cheaper than the corresponding full price for OnDemand instances (up to 90% cheaper!). There is one gotcha: AWS can decommission your Spot instances at any point in time. Your software must be fault-tolerant to leverage this opportunity. 3Assuming you wrote it recently. Your past self from one year ago counts as a stranger for all intents and purposes in the world of software development. Pray that your past self wrote comments for your present self if you are about to pick up again an old project of yours.

📄 Page 18

CONTENTS xvii Zero To Production will therefore put a strong emphasis on test-driven development and continuous integ- ration from the get-go - we will have a CI pipeline set up before we even have a web server up and running! We will be covering techniques such as black-box testing for APIs andHTTPmocking - not wildly popular or well documented in the Rust community yet extremely powerful. We will also borrow terminology and techniques from the Domain Driven Design world, combining them with type-driven design to ensure the correctness of our systems. Our main focus is enterprise software: correct code which is expressive enough to model the domain and supple enough to support its evolution over time. We will thus have a bias for boring and correct solutions, even if they incur a performance overhead that could be optimised away with a more careful and chiseled approach. Get it to run first, optimise it later (if needed). Who Is This Book For TheRust ecosystemhas had a remarkable focus on smashing adoption barriers with amazingmaterial geared towards beginners and newcomers, a relentless effort that goes from documentation to the continuous pol- ishing of the compiler diagnostics. There is value in serving the largest possible audience. At the same time, trying to always speak to everybody can have harmful side-effects: material that would be relevant to intermediate and advanced users but definitely too much too soon for beginners ends up being neglected. I struggled with it first-hand when I started to play around with async/await. There was a significant gap between the knowledge I needed to be productive and the knowledge I had built reading The Rust Book or working in the Rust numerical ecosystem. I wanted to get an answer to a straight-forward question: Can Rust be a productive language for API development? Yes. But it can take some time to figure out how. That’s why I am writing this book. I am writing this book for the seasoned backend developers who have read The Rust Book and are now trying to port over a couple of simple systems. I am writing this book for the new engineers on my team, a trail to help them make sense of the codebases they will contribute to over the coming weeks and months. I amwriting this book for anichewhoseneeds I believe are currently underservedby the articles and resources available in the Rust ecosystem.

📄 Page 19

xviii CONTENTS I am writing this book for myself a year ago. To socialise the knowledge gained during the journey: what does your toolbox look like if you are usingRust for backend development in 2022? What are the design patterns? Where are the pitfalls? If you do not fit this description but you are working towards it I will domy best to help you on the journey: while we won’t be covering a lot of material directly (e.g. most Rust language features) I will try to provide references and links where needed to help you pick up/brush off those concepts along the way. Let’s get started.

📄 Page 20

Chapter 1 Getting Started There is more to a programming language than the language itself: tooling is a key element of the experience of using the language. The same applies tomany other technologies (e.g. RPC frameworks like gRPCorApacheAvro) and it often has a disproportionate impact on the uptake (or the demise) of the technology itself. Tooling should therefore be treated as a first-class concern both when designing and teaching the language itself. The Rust community has put tooling at the forefront since its early days: it shows. We are now going to take a brief tour of a set of tools and utilities that are going to be useful in our jour- ney. Some of them are officially supported by the Rust organisation, others are built and maintained by the community. 1.1 Installing The Rust Toolchain There are various ways to install Rust on your system, but we are going to focus on the recommended path: via rustup. Instructions on how to install rustup itself can be found at https://rustup.rs. rustup is more than a Rust installer - its main value proposition is toolchain management. A toolchain is the combination of a compilation target and a release channel. 1.1.1 Compilation Targets The main purpose of the Rust compiler is to convert Rust code into machine code - a set of instructions that your CPU and operating system can understand and execute. Therefore you need a different backend of the Rust compiler for each compilation target, i.e. for each plat- form (e.g. 64-bit Linux or 64-bit OSX) you want to produce a running executable for. The Rust project strives to support a broad range of compilation targets with various level of guarantees. Targets are split into tiers, from “guaranteed-to-work” Tier 1 to “best-effort” Tier 3. An exhaustive and up-to-date list can be found on the Rust forge website. 1