Web Development With Go (Jon Calhoun) (z-library.sk, 1lib.sk, z-lib.sk)

2

Web Development with Go Learn to Create Real World Web Applications using Go Jonathan Calhoun

ii

Contents About the author v Copyright and license vii The Book Cover ix 1 Introduction 1 1.1 Who is this book for? . . . . . . . . . . . . . . . . . . . . . . 1 1.2 How to use this book . . . . . . . . . . . . . . . . . . . . . . 2 1.3 Reference materials . . . . . . . . . . . . . . . . . . . . . . . 3 1.4 What if I am already an expert developer? . . . . . . . . . . . 3 1.5 What are we building? . . . . . . . . . . . . . . . . . . . . . 4 1.6 Conventions used in this book . . . . . . . . . . . . . . . . . 5 1.6.1 Command-line commands are prefixed with a $ . . . . 5 1.6.2 The subl or ‘atom’ command means “open with your text editor” . . . . . . . . . . . . . . . . . . . . . . . 6 iii

iv CONTENTS 1.6.3 Many code listings will be shortened for clarity . . . . 6 1.6.4 All code samples are limited to 60 columns when possible 7 1.7 Accessing the code . . . . . . . . . . . . . . . . . . . . . . . 7 1.7.1 Why git? . . . . . . . . . . . . . . . . . . . . . . . . 8 1.8 Disclaimer: Not everything is one size fits all . . . . . . . . . 8 1.9 Commenting your exported types . . . . . . . . . . . . . . . . 9 2 A basic web application 11 2.1 Building the server . . . . . . . . . . . . . . . . . . . . . . . 11 2.2 Demystifying our app . . . . . . . . . . . . . . . . . . . . . . 14 3 Adding new pages 23 3.1 Routing with if/else statements . . . . . . . . . . . . . . . . . 24 3.2 Popular routers . . . . . . . . . . . . . . . . . . . . . . . . . 28 3.2.1 net/http.ServeMux . . . . . . . . . . . . . . . . . . . 28 3.2.2 github.com/julienschmidt/httprouter . . . . . . . . . . 28 3.2.3 github.com/gorilla/mux.Router . . . . . . . . . . . . . 29 3.2.4 What about <some other router> . . . . . . . . . . . . 29 3.3 Using the gorilla/mux router . . . . . . . . . . . . . . . . . . 30 3.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 3.4.1 Ex1 - Add an FAQ page . . . . . . . . . . . . . . . . 33 3.4.2 Ex2 - Custom 404 page . . . . . . . . . . . . . . . . . 34

CONTENTS v 3.4.3 Ex3 - [HARD] Try out another router . . . . . . . . . 34 4 A brief introduction to templates 35 4.1 What are templates? . . . . . . . . . . . . . . . . . . . . . . . 36 4.2 Why do we use templates? . . . . . . . . . . . . . . . . . . . 37 4.3 Templates in Go . . . . . . . . . . . . . . . . . . . . . . . . . 37 4.4 Creating a template . . . . . . . . . . . . . . . . . . . . . . . 39 4.5 Contextual encoding . . . . . . . . . . . . . . . . . . . . . . 43 4.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 4.6.1 Ex1 - Add a new template variable . . . . . . . . . . . 47 4.6.2 Ex2 - Experiment with different data types . . . . . . 47 4.6.3 Ex3 - [HARD] Learn how to use nested data . . . . . 48 4.6.4 Ex4 - [HARD] Create an if/else statement in your tem- plate . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 5 Understanding MVC 49 5.1 Model-View-Controller (MVC) . . . . . . . . . . . . . . . . . 49 5.2 Walking through a web request . . . . . . . . . . . . . . . . . 51 5.3 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 5.3.1 Ex1 - What does MVC stand for? . . . . . . . . . . . 55 5.3.2 Ex2 - What is each layer of MVC responsible for? . . 56 5.3.3 Ex3 - What are some benefits to using MVC? . . . . . 56

vi CONTENTS 6 Creating our first views 57 6.1 The home template . . . . . . . . . . . . . . . . . . . . . . . 58 6.2 The contact template . . . . . . . . . . . . . . . . . . . . . . 61 6.3 Creating a reusable Bootstrap layout . . . . . . . . . . . . . . 64 6.3.1 Named templates . . . . . . . . . . . . . . . . . . . . 65 6.3.2 Creating a view type . . . . . . . . . . . . . . . . . . 69 6.3.3 Creating the Bootstrap layout . . . . . . . . . . . . . 74 6.4 Adding a navigation bar . . . . . . . . . . . . . . . . . . . . . 81 6.5 Cleaning up our code . . . . . . . . . . . . . . . . . . . . . . 85 6.5.1 What is globbing? . . . . . . . . . . . . . . . . . . . 86 6.5.2 Using filepath.Glob . . . . . . . . . . . . . . . . 87 6.5.3 Simplifying view rendering . . . . . . . . . . . . . . 92 6.5.4 Moving our footer to the layout . . . . . . . . . . . . 94 6.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 6.6.1 Ex1 - Create an FAQ page with the Bootstrap layout . 96 6.6.2 Ex2 - Update the navbar to link to the FAQ page . . . 96 6.6.3 Ex3 - Create a new layout . . . . . . . . . . . . . . . 97 7 Creating a sign up page 99 7.1 Add a sign up page with a form . . . . . . . . . . . . . . . . . 99 7.1.1 Creating a Bootstrap sign up form . . . . . . . . . . . 100

CONTENTS vii 7.1.2 Wrapping our form in a panel . . . . . . . . . . . . . 106 7.1.3 Adding the sign up link to our navbar . . . . . . . . . 112 7.2 An intro to REST . . . . . . . . . . . . . . . . . . . . . . . . 115 7.2.1 How REST affects our code . . . . . . . . . . . . . . 116 7.3 Creating our first controller . . . . . . . . . . . . . . . . . . . 117 7.3.1 Create the users controller . . . . . . . . . . . . . . . 119 7.3.2 Moving the sign up page code . . . . . . . . . . . . . 120 7.3.3 Connecting our router and the users controller together 125 7.4 Processing the sign up form . . . . . . . . . . . . . . . . . . . 128 7.4.1 Stubbing the create user action . . . . . . . . . . . . . 129 7.4.2 HTTP request methods and gorilla/mux . . . . . . . . 130 7.4.3 Parsing a POST form . . . . . . . . . . . . . . . . . . 134 7.4.4 Parsing forms with gorilla/schema . . . . . . . . 137 7.4.5 Keeping our parsing code DRY . . . . . . . . . . . . 141 7.5 Cleaning up and creating a static controller . . . . . . . . . . 146 7.5.1 Creating the static controller . . . . . . . . . . . . . . 146 7.5.2 Simplifying the creation of new views . . . . . . . . . 151 7.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 7.6.1 Ex1 - Add the FAQ page to the static controller . . . . 157 7.6.2 Ex2 - Create a new controller for galleries . . . . . . . 157 7.6.3 Exercise cleanup . . . . . . . . . . . . . . . . . . . . 158

viii CONTENTS 8 An introduction to databases 159 8.1 Our web app will use PostgreSQL . . . . . . . . . . . . . . . 160 8.2 Setting up PostgreSQL . . . . . . . . . . . . . . . . . . . . . 162 8.2.1 Install PostgreSQL . . . . . . . . . . . . . . . . . . . 162 8.2.2 Learn how to connect to Postgres . . . . . . . . . . . 163 8.2.3 Learn the basics of SQL . . . . . . . . . . . . . . . . 164 8.2.4 Gather information needed to connect to your Postgres install . . . . . . . . . . . . . . . . . . . . . . . . . . 165 8.3 Using Postgres with Go and raw SQL . . . . . . . . . . . . . 167 8.3.1 Connecting to Postgres with the database/sql package . 167 8.3.2 Creating SQL tables to test with . . . . . . . . . . . . 173 8.3.3 Writing records with database/sql . . . . . . . . . . . 174 8.3.4 Querying a single record with database/sql . . . . . . 176 8.3.5 Querying multiple records with database/sql . . . . . . 178 8.3.6 Writing a relational record . . . . . . . . . . . . . . . 181 8.3.7 Querying related records . . . . . . . . . . . . . . . . 183 8.3.8 Delete the SQL tables we were testing with . . . . . . 184 8.4 Using GORM to interact with a database . . . . . . . . . . . . 185 8.4.1 Installing GORM and connecting to a database . . . . 185 8.4.2 Defining a GORM model . . . . . . . . . . . . . . . . 187 8.4.3 Creating and migrating tables with GORM . . . . . . 189

CONTENTS ix 8.4.4 Logging with GORM . . . . . . . . . . . . . . . . . . 191 8.4.5 Creating a record with GORM . . . . . . . . . . . . . 193 8.4.6 Querying a single record with GORM . . . . . . . . . 196 8.4.7 Querying multiple records with GORM . . . . . . . . 200 8.4.8 Creating related models with GORM . . . . . . . . . 201 8.4.9 Querying relational data with GORM . . . . . . . . . 204 8.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206 8.5.1 Ex1 - What changes won’t the AutoMigrate function provided by GORM handle for you? . . . . . . . . . . 206 8.5.2 Ex2 - What is gorm.Model used for? . . . . . . . . . 206 8.5.3 Ex3 - Experiment using a few more GORM methods. . 207 8.5.4 Ex4 - Experiment with query chaining . . . . . . . . . 207 8.5.5 Ex5 - Learn to execute raw SQL with GORM . . . . . 207 9 Creating the user model 209 9.1 Defining a User type . . . . . . . . . . . . . . . . . . . . . . 209 9.2 Creating the UserService interface and querying for users . 213 9.3 Creating users . . . . . . . . . . . . . . . . . . . . . . . . . . 219 9.4 Querying by email and DRYing up our code . . . . . . . . . . 221 9.5 Updating and deleting users . . . . . . . . . . . . . . . . . . . 226 9.6 AutoMigrating and returning errors from DestructiveReset . . 229 9.7 Connecting the user service and controller . . . . . . . . . . . 231

x CONTENTS 9.7.1 Adding the name field the sign up form . . . . . . . . 231 9.7.2 Setting up a user service in our web application . . . . 234 9.7.3 Using the users service in our users controller . . . . . 237 9.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240 9.8.1 Ex1 - Add an Age field to our user resource . . . . . . 241 9.8.2 Ex2 - Write a method to query the first user with a spe- cific age . . . . . . . . . . . . . . . . . . . . . . . . . 241 9.8.3 Ex3 - [HARD] Write a method to query many users by age range . . . . . . . . . . . . . . . . . . . . . . . . 241 10 Building an authentication system 243 10.1 Why not use another package or service? . . . . . . . . . . . . 244 10.2 Secure your server with SSL/TLS . . . . . . . . . . . . . . . 245 10.3 Hash passwords properly . . . . . . . . . . . . . . . . . . . . 245 10.3.1 What is a hash function? . . . . . . . . . . . . . . . . 246 10.3.2 Store hashed passwords, not raw passwords . . . . . . 247 10.3.3 Salt and pepper passwords . . . . . . . . . . . . . . . 248 10.4 Implementing password hashing for our users . . . . . . . . . 252 10.4.1 Adding password fields to the user model . . . . . . . 252 10.4.2 Hash passwords with bcrypt before saving . . . . . . . 254 10.4.3 Retrieving passwords from the sign up form . . . . . . 258 10.4.4 Salting and peppering passwords . . . . . . . . . . . . 259

CONTENTS xi 10.5 Authenticating returning users . . . . . . . . . . . . . . . . . 262 10.5.1 Creating the login template . . . . . . . . . . . . . . . 262 10.5.2 Creating the login action . . . . . . . . . . . . . . . . 264 10.5.3 Implementing the Authenticate method . . . . . . 267 10.5.4 Calling Authenticate from our login action . . . . 271 10.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272 10.6.1 Ex1 - What is a hash function? . . . . . . . . . . . . . 272 10.6.2 Ex2 - What purpose does a salt and pepper serve? . . . 272 10.6.3 Ex3 - Timing attacks . . . . . . . . . . . . . . . . . . 273 11 Remembering users 275 11.1 What are cookies . . . . . . . . . . . . . . . . . . . . . . . . 276 11.2 Creating our first cookie . . . . . . . . . . . . . . . . . . . . 277 11.3 Viewing cookies . . . . . . . . . . . . . . . . . . . . . . . . . 281 11.3.1 Viewing cookies in Google Chrome . . . . . . . . . . 281 11.3.2 Viewing cookies with Go code . . . . . . . . . . . . . 287 11.4 Securing our cookies from tampering . . . . . . . . . . . . . . 289 11.4.1 Digitally signing data . . . . . . . . . . . . . . . . . . 289 11.4.2 Obfuscating cookie data . . . . . . . . . . . . . . . . 292 11.5 Generating remember tokens . . . . . . . . . . . . . . . . . . 293 11.5.1 Why do we use 32 bytes? . . . . . . . . . . . . . . . . 299

xii CONTENTS 11.6 Hashing remember tokens . . . . . . . . . . . . . . . . . . . 301 11.6.1 How to use the hash package . . . . . . . . . . . . . 302 11.6.2 Using the crypto/hmac package . . . . . . . . . . . 304 11.6.3 Writing our own hash package . . . . . . . . . . . . 305 11.7 Hashing remember tokens in the user service . . . . . . . . . 308 11.7.1 Adding remember token fields to our User type . . . . 309 11.7.2 Setting a remember token when a user is created . . . 310 11.7.3 Adding an HMAC field to the UserService . . . . . . 312 11.7.4 Hashing remember tokens on create and update . . . . 314 11.7.5 Retrieving a user by remember token . . . . . . . . . 315 11.7.6 Resetting our DB and testing . . . . . . . . . . . . . . 316 11.8 Remembering users . . . . . . . . . . . . . . . . . . . . . . . 319 11.8.1 Storing remember tokens in cookies . . . . . . . . . . 320 11.8.2 Restricting page access . . . . . . . . . . . . . . . . . 324 11.9 Securing our cookies from XSS . . . . . . . . . . . . . . . . 326 11.10Securing our cookies from theft . . . . . . . . . . . . . . . . 328 11.11Preventing CSRF attacks . . . . . . . . . . . . . . . . . . . . 330 11.12Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330 11.12.1 Ex1 - Experiment with redirection . . . . . . . . . . . 331 11.12.2 Ex2 - Create cookies with different data . . . . . . . . 331 11.12.3 Ex3 - [HARD] Experiment with the hash package . . . 331

CONTENTS xiii 12 Normalizing and validating data 333 12.1 Separating responsibilities . . . . . . . . . . . . . . . . . . . 336 12.1.1 Rethinking the models package . . . . . . . . . . . . 337 12.1.2 Static types vs interfaces . . . . . . . . . . . . . . . . 342 12.2 The UserDB interface . . . . . . . . . . . . . . . . . . . . . . 351 12.3 The UserService interface . . . . . . . . . . . . . . . . . . . . 361 12.4 Writing and organizing validation code . . . . . . . . . . . . . 364 12.4.1 The straightforward approach . . . . . . . . . . . . . 365 12.4.2 A few more validation examples . . . . . . . . . . . . 368 12.4.3 Reusable validations . . . . . . . . . . . . . . . . . . 371 12.5 Writing validators and normalizers . . . . . . . . . . . . . . . 377 12.5.1 Remember token normalizer . . . . . . . . . . . . . . 377 12.5.2 Ensuring remember tokens are set on create . . . . . . 380 12.5.3 Ensuring a valid ID on delete . . . . . . . . . . . . . . 382 12.6 Validating and normalizing email addresses . . . . . . . . . . 386 12.6.1 Converting emails to lowercase and trimming whitespace386 12.6.2 Requiring email addresses . . . . . . . . . . . . . . . 390 12.6.3 Verifying emails match a pattern . . . . . . . . . . . . 392 12.6.4 Verifying an email address isn’t taken . . . . . . . . . 401 12.7 Cleaning up our error naming . . . . . . . . . . . . . . . . . . 404 12.8 Validating and normalizing passwords . . . . . . . . . . . . . 407

xiv CONTENTS 12.8.1 Verifying passwords have a minimum length . . . . . 408 12.8.2 Requiring a password . . . . . . . . . . . . . . . . . . 409 12.9 Validating and normalizing remember tokens . . . . . . . . . 412 13 Displaying errors to the end user 417 13.1 Rendering alerts in the UI . . . . . . . . . . . . . . . . . . . . 418 13.2 Rendering dynamic alerts . . . . . . . . . . . . . . . . . . . . 421 13.3 Only display alerts when we need them . . . . . . . . . . . . 425 13.4 A more permanent data type for views . . . . . . . . . . . . . 429 13.5 Handling errors in the sign up form . . . . . . . . . . . . . . . 433 13.6 White-listing error messages . . . . . . . . . . . . . . . . . . 439 13.7 Handling login errors . . . . . . . . . . . . . . . . . . . . . . 447 13.8 Recovering from view rendering errors . . . . . . . . . . . . . 450 14 Creating the gallery resource 455 14.1 Defining the gallery model . . . . . . . . . . . . . . . . . . . 456 14.2 Introducing the GalleryService . . . . . . . . . . . . . . . . . 458 14.3 Constructing many services . . . . . . . . . . . . . . . . . . . 459 14.4 Closing and migrating all models . . . . . . . . . . . . . . . . 464 14.5 Creating new galleries . . . . . . . . . . . . . . . . . . . . . 468 14.5.1 Implementing the gallery service . . . . . . . . . . . . 469 14.5.2 The galleries controller . . . . . . . . . . . . . . . . . 471

CONTENTS xv 14.5.3 Processing the new gallery form . . . . . . . . . . . . 475 14.5.4 Validating galleries . . . . . . . . . . . . . . . . . . . 478 14.6 Requiring users via middleware . . . . . . . . . . . . . . . . 480 14.6.1 Creating our first middleware . . . . . . . . . . . . . 481 14.6.2 Storing request-scoped data with context . . . . . . . 488 14.7 Displaying galleries . . . . . . . . . . . . . . . . . . . . . . . 496 14.7.1 Creating the show gallery view . . . . . . . . . . . . . 497 14.7.2 Parsing the gallery ID from the path . . . . . . . . . . 498 14.7.3 Looking up galleries by ID . . . . . . . . . . . . . . . 503 14.7.4 Generating URLs with params . . . . . . . . . . . . . 506 14.8 Editing galleries . . . . . . . . . . . . . . . . . . . . . . . . . 511 14.8.1 The edit gallery action . . . . . . . . . . . . . . . . . 512 14.8.2 Parsing the edit gallery form . . . . . . . . . . . . . . 517 14.8.3 Updating gallery models . . . . . . . . . . . . . . . . 520 14.9 Deleting galleries . . . . . . . . . . . . . . . . . . . . . . . . 522 14.10Viewing all owned galleries . . . . . . . . . . . . . . . . . . . 526 14.10.1 Querying galleries by user ID . . . . . . . . . . . . . 527 14.10.2 Adding the Index handler . . . . . . . . . . . . . . . . 529 14.10.3 Iterating over slices in Go templates . . . . . . . . . . 531 14.11Improving the user experience and cleaning up . . . . . . . . 536 14.11.1 Adding intuitive redirects . . . . . . . . . . . . . . . . 537

xvi CONTENTS 14.11.2 Navigation for signed in users . . . . . . . . . . . . . 542 15 Adding images to galleries 551 15.1 Image upload form . . . . . . . . . . . . . . . . . . . . . . . 552 15.2 Processing image uploads . . . . . . . . . . . . . . . . . . . . 557 15.3 Creating the image service . . . . . . . . . . . . . . . . . . . 569 15.4 Looking up images by gallery ID . . . . . . . . . . . . . . . . 575 15.5 Serving static files in Go . . . . . . . . . . . . . . . . . . . . 580 15.6 Rendering images in columns . . . . . . . . . . . . . . . . . . 584 15.7 Letting users delete images . . . . . . . . . . . . . . . . . . . 592 15.8 Known Bugs . . . . . . . . . . . . . . . . . . . . . . . . . . 599 16 Deploying to production 601 16.1 Error handling . . . . . . . . . . . . . . . . . . . . . . . . . . 602 16.2 Serving static assets . . . . . . . . . . . . . . . . . . . . . . . 603 16.3 CSRF protection . . . . . . . . . . . . . . . . . . . . . . . . 606 16.4 Limiting middleware for static assets . . . . . . . . . . . . . . 615 16.5 Fixing bugs . . . . . . . . . . . . . . . . . . . . . . . . . . . 616 16.5.1 URL encoding image paths . . . . . . . . . . . . . . . 617 16.5.2 Redirecting after image uploads . . . . . . . . . . . . 620 16.6 Configuring our application . . . . . . . . . . . . . . . . . . . 622 16.6.1 Finding variables that need provided . . . . . . . . . . 622

CONTENTS xvii 16.6.2 Using functional options . . . . . . . . . . . . . . . . 627 16.6.3 JSON configuration files . . . . . . . . . . . . . . . . 636 16.7 Setting up a server . . . . . . . . . . . . . . . . . . . . . . . 644 16.7.1 Digital Ocean droplet . . . . . . . . . . . . . . . . . . 644 16.7.2 Installing PostgreSQL in production . . . . . . . . . . 647 16.7.3 Installing Go . . . . . . . . . . . . . . . . . . . . . . 652 16.7.4 Setting up Caddy . . . . . . . . . . . . . . . . . . . . 654 16.7.5 Creating a service for our app . . . . . . . . . . . . . 660 16.7.6 Setting up a production config . . . . . . . . . . . . . 661 16.7.7 Creating a deploy script . . . . . . . . . . . . . . . . 662 17 Filling in the gaps 669 17.1 Deleting cookies and logging out users . . . . . . . . . . . . . 669 17.2 Redirecting with alerts . . . . . . . . . . . . . . . . . . . . . 672 17.3 Emailing users . . . . . . . . . . . . . . . . . . . . . . . . . 677 17.4 Persisting and prefilling form data . . . . . . . . . . . . . . . 681 17.5 Resetting passwords . . . . . . . . . . . . . . . . . . . . . . . 686 17.5.1 Creating the database model . . . . . . . . . . . . . . 687 17.5.2 Updating the services . . . . . . . . . . . . . . . . . . 692 17.5.3 Forgotten password forms . . . . . . . . . . . . . . . 698 17.5.4 Controller actions and views . . . . . . . . . . . . . . 702

xviii CONTENTS 17.5.5 Emailing users and building URLs . . . . . . . . . . . 707 17.6 Where do I go next? . . . . . . . . . . . . . . . . . . . . . . . 711 18 Appendix 713 18.1 Using interfaces in Go . . . . . . . . . . . . . . . . . . . . . 713 18.2 Refactoring with gorename . . . . . . . . . . . . . . . . . . . 716 18.3 Handling Bootstrap issues . . . . . . . . . . . . . . . . . . . 719 18.4 Private model errors . . . . . . . . . . . . . . . . . . . . . . . 719