Go Cookbook (Aaron Torres) (z-library.sk, 1lib.sk, z-lib.sk)

📄 File Format: PDF

💾 File Size: 1.9 MB

📖 Read Online ⬇️ Download

Registered users can read the full content for free

Register as a Gaohf Library member to read the complete e-book online for free and enjoy a better reading experience.

📄 Page 1

(This page has no text content)

📄 Page 2

Go Cookbook Build modular, readable, and testable applications in Go Aaron Torres BIRMINGHAM - MUMBAI

📄 Page 3

Go Cookbook Copyright © 2017 Packt Publishing All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information. First published: June 2017 Production reference: 1240617 Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK. ISBN 978-1-78328-683-6 www.packtpub.com

📄 Page 4

Credits Author Aaron Torres Copy Editor Pranjali Chury Reviewer Julien Da Silva Project Coordinator Vaidehi Sawant Commissioning Editor Kunal Parikh Proofreader Safis Editing Acquisition Editor Karan Sadawana Indexer Tejal Daruwale Soni Content Development Editor Rohit Kumar Singh Graphics Jason Monteiro Technical Editor Vivek Pala Production Coordinator Nilesh Mohite

📄 Page 5

About the Author Aaron Torres received his master’s of science degree in computer science from New Mexico Institute of Mining and Technology. He has worked on distributed systems in high performance computing and in large-scale web and microservices applications. He currently leads a team of Go developers that refines and focuses on Go best practices with an emphasis on continuous delivery and automated testing. Aaron has published a number of papers and has several patents in the area of storage and I/O. He is passionate about sharing his knowledge and ideas with others. He is also a huge fan of the Go language and open source for backend systems and development.

📄 Page 6

About the Reviewer Julien Da Silva is a software engineer and architect specializing in scalable, distributed systems. Previously at Hailo, he was a part of the team that built its Golang platform, which is widely recognized as one of the early successful implementations of microservices. Hailo was later acquired by MyTaxi in 2016. He is currently working as a core architect at LastMileLink, part of CitySprint group, UK leader in same day courier services.

📄 Page 7

www.PacktPub.com For support files and downloads related to your book, please visit www.PacktPub.com. Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at service@packtpub.com for more details. At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks. h t t p s ://w w w . p a c k t p u b . c o m /m a p t Get the most in-demand software skills with Mapt. Mapt gives you full access to all Packt books and video courses, as well as industry-leading tools to help you plan your personal development and advance your career. Why subscribe? Fully searchable across every book published by Packt Copy and paste, print, and bookmark content On demand and accessible via a web browser

📄 Page 8

Customer Feedback Thanks for purchasing this Packt book. At Packt, quality is at the heart of our editorial process. To help us improve, please leave us an honest review on this book's Amazon page at h t t p s ://w w w . a m a z o n . c o m /G o - C o o k b o o k - A a r o n - T o r r e s - e b o o k /d p /B 01M S 2M O F P /r e f =s r _ 1_ 16?i e =U T F 8&q i d =1497940919&s r =8- 16&k e y w o r d s =g o +c o o k b o o k . If you'd like to join our team of regular reviewers, you can e-mail us at customerreviews@packtpub.com. We award our regular reviewers with free eBooks and videos in exchange for their valuable feedback. Help us be relentless in improving our products!

📄 Page 9

Table of Contents Preface 1 Chapter 1: I/O and File Systems 7 Introduction 7 Using the common I/O interfaces 8 Getting ready 8 How to do it... 9 How it works... 12 Using the bytes and strings packages 12 Getting ready 12 How to do it... 12 How it works... 16 Working with directories and files 16 Getting ready 17 How to do it... 17 How it works... 20 Working with the CSV format 21 Getting ready 21 How to do it... 21 How it works... 26 Working with temporary files 26 Getting ready 26 How to do it... 27 How it works... 28 Working with text/template and HTML/templates 28 Getting ready 29 How to do it... 29 How it works... 35 Chapter 2: Command-Line Tools 36 Introduction 36 Using command-line flags 37 Getting ready 37 How to do it... 37 How it works... 41 Using command-line arguments 41

📄 Page 10

[ ii ] Getting ready 41 How to do it... 41 How it works... 45 Reading and setting environment variables 45 Getting ready 45 How to do it... 45 How it works... 49 Configuration using TOML, YAML, and JSON 49 Getting ready 49 How to do it... 50 How it works... 55 Working with Unix pipes 56 Getting ready 56 How to do it... 56 How it works... 58 Catching and handling signals 58 Getting ready 58 How to do it... 59 How it works... 61 An ANSI coloring application 61 Getting ready 61 How to do it... 61 How it works... 64 Chapter 3: Data Conversion and Composition 65 Introduction 65 Converting data types and interface casting 66 Getting ready 66 How to do it... 66 How it works... 70 Working with numeric data types using math and math/big 70 Getting ready 70 How to do it... 71 How it works... 74 Currency conversions and float64 considerations 74 Getting ready 74 How to do it... 74 How it works... 78 Using pointers and SQL NullTypes for encoding and decoding 78 Getting ready 78

📄 Page 11

[ iii ] How to do it... 79 How it works... 84 Encoding and decoding Go data 84 Getting ready 84 How to do it... 84 How it works... 88 Struct tags and basic reflection in Go 88 Getting ready 89 How to do it... 89 How it works... 94 Implementing collections via closures 94 Getting ready 94 How to do it... 95 How it works... 98 Chapter 4: Error Handling in Go 99 Introduction 99 Handling errors and the Error interface 100 Getting ready 100 How to do it... 100 How it works... 103 Using the pkg/errors package and wrapping errors 103 Getting ready 103 How to do it... 103 How it works... 106 Using the log package and understanding when to log errors 107 Getting ready 107 How to do it... 107 How it works... 110 Structured logging with the apex and logrus packages 110 Getting ready 111 How to do it... 111 How it works... 114 Logging with the context package 114 Getting ready 114 How to do it... 114 How it works... 117 Using package-level global variables 118 Getting ready 118 How to do it... 118

📄 Page 12

[ iv ] How it works... 121 Catching panics for long running processes 121 Getting ready 121 How to do it... 121 How it works... 123 Chapter 5: All about Databases and Storage 124 Introduction 124 The database/sql package with MySQL 125 Getting ready 125 How to do it... 125 How it works... 129 Executing a database transaction interface 129 Getting ready 130 How to do it... 130 How it works... 134 Connection pooling, rate limiting, and timeouts for SQL 134 Getting ready 134 How to do it... 134 How it works... 137 Working with Redis 137 Getting ready 137 How to do it... 138 How it works... 141 Using NoSQL with MongoDB and mgo 141 Getting ready 142 How to do it... 142 How it works... 144 Creating storage interfaces for data portability 145 Getting ready 145 How to do it... 145 How it works... 148 Chapter 6: Web Clients and APIs 150 Introduction 150 Initializing, storing, and passing http.Client structs 151 Getting ready 151 How to do it... 151 How it works... 155 Writing a client for a REST API 155

📄 Page 13

[ v ] Getting ready 155 How to do it... 156 How it works... 158 Executing parallel and async client requests 159 Getting ready 159 How to do it... 159 How it works... 161 Making use of OAuth2 clients 162 Getting ready 162 How to do it... 162 How it works... 165 Implementing an OAuth2 token storage interface 166 Getting ready 166 How to do it... 166 How it works... 171 Wrapping a client in added functionality and function composition 172 Getting ready 172 How to do it... 173 How it works... 176 Understanding GRPC clients 177 Getting ready 177 How to do it... 177 How it works... 181 Chapter 7: Microservices for Applications in Go 182 Introduction 182 Working with web handlers, requests, and ResponseWriters 183 Getting ready 183 How to do it... 184 How it works... 186 Using structs and closures for stateful handlers 187 Getting ready 187 How to do it... 187 How it works... 191 Validating input for Go structs and user inputs 192 Getting ready 192 How to do it... 192 How it works... 196 Rendering and content negotiation 196 Getting ready 196

📄 Page 14

[ vi ] How to do it... 197 How it works... 199 Implementing and using middleware 200 Getting ready 200 How to do it... 200 How it works... 204 Building a reverse proxy application 204 Getting ready 204 How to do it... 205 How it works... 208 Exporting GRPC as a JSON API 208 Getting ready 208 How to do it... 208 How it works... 214 Chapter 8: Testing 215 Introduction 215 Mocking using the standard library 216 Getting ready 216 How to do it... 216 How it works... 220 Using the Mockgen package 220 Getting ready 220 How to do it... 220 How it works... 224 Using table-driven tests to improve coverage 224 Getting ready 225 How to do it... 225 How it works... 227 Using third-party testing tools 227 Getting ready 227 How to do it... 228 How it works... 231 Practical fuzzing 232 Getting ready 232 How to do it... 232 How it works... 235 Behavior testing using Go 236 Getting ready 236 How to do it... 236

📄 Page 15

[ vii ] How it works... 240 Chapter 9: Parallelism and Concurrency 241 Introduction 241 Using channels and the select statement 242 Getting ready 242 How to do it... 242 How it works... 245 Performing async operations with sync.WaitGroup 245 Getting ready 245 How to do it... 246 How it works... 249 Using atomic operations and mutex 249 Getting ready 249 How to do it... 250 How it works... 253 Using the context package 253 Getting ready 254 How to do it... 254 How it works... 256 Executing state management for channels 257 Getting ready 257 How to do it... 257 How it works... 261 Using the worker pool design pattern 261 Getting ready 261 How to do it... 261 How it works... 265 Using workers to create pipelines 266 Getting ready 266 How to do it... 266 How it works... 271 Chapter 10: Distributed Systems 272 Introduction 272 Using service discovery with Consul 273 Getting ready 273 How to do it... 273 How it works... 276 Implementing basic consensus using Raft 277

📄 Page 16

[ viii ] Getting ready 277 How to do it... 277 How it works... 283 Using containerization with Docker 283 Getting ready 284 How to do it... 284 How it works... 287 Orchestration and deployment strategies 288 Getting ready 288 How to do it... 288 How it works... 290 Monitoring applications 291 Getting ready 291 How to do it... 291 How it works... 294 Collecting metrics 294 Getting ready 294 How to do it... 294 How it works... 297 Chapter 11: Reactive Programming and Data Streams 298 Introduction 298 Goflow for dataflow programming 299 Getting ready 299 How to do it... 299 How it works... 302 Reactive programming with RxGo 303 Getting ready 303 How to do it... 303 How it works... 306 Using Kafka with Sarama 306 Getting ready 306 How to do it... 307 How it works... 309 Using async producers with Kafka 310 Getting ready 310 How to do it... 310 How it works... 313 Connecting Kafka to Goflow 313 Getting ready 314

📄 Page 17

[ ix ] How to do it... 314 How it works... 317 Writing a GraphQL server in Go 318 Getting ready 318 How to do it... 318 How it works... 324 Chapter 12: Serverless Programming 325 Introduction 325 Go programming on Lambda with Apex 325 Getting ready 326 How to do it... 326 How it works... 329 Apex serverless logging and metrics 330 Getting ready 330 How to do it... 330 How it works... 333 Google App Engine with Go 334 Getting ready 334 How to do it... 335 How it works... 338 Working with Firebase using zabawaba99/firego 339 Getting ready 339 How to do it... 340 How it works... 342 Chapter 13: Performance Improvements, Tips, and Tricks 343 Introduction 343 Speeding up compilation and testing cycles 344 Getting ready 344 How to do it... 344 How it works... 346 Using the pprof tool 347 Getting ready 347 How to do it... 347 How it works... 352 Benchmarking and finding bottlenecks 352 Getting ready 352 How to do it... 353 How it works... 356

📄 Page 18

[ x ] Memory allocation and heap management 356 Getting ready 356 How to do it... 356 How it works... 359 Vendoring and project layout 360 Getting ready 360 How to do it... 360 How it works... 365 Using fasthttprouter and fasthttp 366 Getting ready 366 How to do it... 366 How it works... 369 Index 370

📄 Page 19

Preface Thank you for choosing this book! I hope it will be a handy reference for developers to quickly look up Go development patterns. It is meant to be a companion to other resources and a reference that will hopefully be useful long after reading it once. Each recipe in this book includes working, simple, and tested code that can be used as a reference or foundation for your own applications. The book covers a range of content from basic to advanced topics. What this book covers Chapter 1, I/O and File Systems, covers common Go I/O interfaces and explores working with filesystems. This includes temporary files, templates, and CSV files. Chapter 2, Command-Line Tools, looks at taking in user input via a command line and explores processing common datatypes such as TOML, YAML, and JSON. Chapter 3, Data Conversion and Composition, demonstrates methods for casting and converting between Go interfaces and data types. It also showcases encoding strategies and some functional design patterns for Go. Chapter 4, Error Handling in Go, showcases strategies to handle errors in Go. It explores how to pass errors, handle them, and log them. Chapter 5, All about Databases and Storage, deals with various storage libraries for accessing data storage systems such as MySQL. It also demonstrates the use of interfaces to decouple your library from your application logic. Chapter 6, Web Clients and APIs, implements Go HTTP client interfaces, REST clients, OAuth2 clients, decorating and extending clients, and GRPC. Chapter 7, Microservices for Applications in Go, explores web handlers, passing in a state to a handler, validation of user input, and middleware. Chapter 8, Testing, focuses on mocking, test coverage, fuzzing, behavior testing, and helpful testing tools. Chapter 9, Parallelism and Concurrency, provides a reference for channels and async operations, atomic values, Go context objects, and channel state management.

📄 Page 20

Preface [ 2 ] Chapter 10, Distributed Systems, implements service discovery, Docker containerization, metrics and monitoring, and orchestration. It mostly deals with deployment and productionisation of Go applications. Chapter 11, Reactive Programming and Data Streams, explores reactive and dataflow applications, Kafka and distributed message queues, and GraphQL servers. Chapter 12, Serverless Programming, deals with deploying Go applications without maintaining a server. This includes using Google App Engine, Firebase, Lambda, and logging in these serverless environment. Chapter 13, Performance Improvements, Tips, and Tricks, is the final chapter and deals with benchmarking, identifying bottlenecks, optimizing, and improving the HTTP performance for Go applications. What you need for this book To use this book, you'll need the following: A Unix programming environment The latest version of the Go 1.x series An Internet connection Permission to install additional packages as described in each chapter Who this book is for This book is aimed for web developers, programmers, and enterprise developers. Basic knowledge of the Go language is assumed. Experience with backend application development is not necessary, but may help understand the motivation behind some of the recipes. This book serves as a good reference for Go developers who are already proficient but need a quick reminder, example, or reference. With the open source repository, it should be possible to share these examples quickly with a team as well. Sections In this book, you will find several headings that appear frequently (Getting ready, How to do it…, How it works…, There's more…, and See also). To give clear instructions on how to complete a recipe, we use these sections as follows: