📄 Page
1
Sau Sheong Chang Go Cookbook Expert Solutions for Commonly Needed Go Tasks
📄 Page
2
OTHER PROGR AMMING L ANGUAGES “Developers new to Go often want to quickly implement a common task but don’t know the best way to do so. Sau Sheong Chang’s Go Cookbook provides answers to these questions. It’s a good resource for discovering Go’s standard library.” —Jon Bodner Author of Learning Go and Staff Engineer, Datadog Go Cookbook Twitter: @oreillymedia linkedin.com/company/oreilly-media youtube.com/oreillymedia Go is an increasingly popular language for programming everything from web applications to distributed network services. While Go is relatively easy and familiar for most programmers coming from the C/Java tradition, there are enough differences for developers to wonder, How can I do this in Go? This practical guide provides recipes to help you unravel common problems and perform useful tasks when working with Go. Each recipe includes self-contained code solutions that you can freely use, along with a discussion of how and why they work. Programmers new to Go can quickly ramp up their knowledge while accomplishing useful tasks, and experienced Go developers can save time by cutting and pasting proven code directly into their applications. Recipes in this guide will help you: • Create a module • Call code from another module • Return and handle an error • Convert strings to numbers (or convert numbers to strings) • Modify multiple characters in a string • Create substrings from a string • Capture string input • And so much more Sau Sheong Chang, a 28-year veteran of the software development industry, has been involved in building software products in many industries using various technologies. He’s been an active member of the software development communities for Java and Ruby as well as for Go. He runs meetups and gives talks in conferences all around the world. US $79.99 CAN $99.99 ISBN: 978-1-098-12211-9
📄 Page
3
Praise for Go Cookbook Developers new to Go often want to quickly implement a common task but don’t know the best way to do so. Sau Sheong Chang’s Go Cookbook provides answers to these questions. It’s a good resource for discovering Go’s standard library. —Jon Bodner, Staff Engineer, Datadog and author of Learning Go Go Cookbook is an indispensable companion for seasoned programmers looking to unleash the full potential of Go. Its practical recipes, covering everything from error handling and concurrency to networking and testing, and much more, make it a go-to resource for solving practical challenges. A must-have for any Go developer. —Alex Chen, CTO, FMZ Quant This book is a great asset to any Go developers, it covers many daily essential recipes that any go developer would face including module management, logging, string manipulations, file IO, structs and interfaces, http server creation, unit testing and benchmarking. —Jason Wong Ho Chi, Senior Solution Engineer on APM of AppDynamics, Go hobbyist, Elasticsearch specialist, and trainer The tips provided throughout the book are quite practical for production use, and the organization of chapters into problem-solution subsections along with the code snippets makes topics like testing and benchmarking easy to comprehend. A must-read for entry-level software engineers. —Vaibhav Jain, Senior Software Engineer, Ninja Van (and currently a Master’s student, Georgia Institute of Technology)
📄 Page
4
Sau Sheong’s Go Cookbook takes the developer on a brief yet rich tour through some of Go’s more intricate areas. Actionable examples coupled with clear explanations quickly empower the developer to adopt alternative approaches, libraries and techniques. The book is relevant for a wide range of backgrounds from beginner to experts alike wanting to brush up on recent language features critically including coverage and fuzzing. —Daniel J Blueman, Principal Software Engineer, Numascale AS Go Cookbook by Sau Sheong Chang is a great resource for those who want to learn Go quickly. It covers essential topics in a straightforward manner. I personally found it to be very useful as a reference guide. —Carlos Alexandre Queiroz, Global Head of Data Science Engineering at a global bank
📄 Page
5
Sau Sheong Chang Go Cookbook Expert Solutions for Commonly Needed Go Tasks
📄 Page
6
978-1-098-12211-9 [LSI] Go Cookbook by Sau Sheong Chang Copyright © 2023 Sau Sheong Chang. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com. Acquisitions Editors: Zan McQuade; Brian Guerin Development Editor: Shira Evans Production Editor: Elizabeth Faerm Copyeditor: Kim Cofer Proofreader: Piper Editorial Consulting, LLC Indexer: Potomac Indexing, LLC Interior Designer: David Futato Cover Designer: Karen Montgomery Illustrator: Kate Dullea September 2023: First Edition Revision History for the First Edition 2023-09-13: First Release See http://oreilly.com/catalog/errata.csp?isbn=9781098122119 for release details. The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Go Cookbook, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc. The views expressed in this work are those of the author, and do not represent the publisher’s views. While the publisher and the author have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights.
📄 Page
7
Table of Contents Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi 1. Getting Started Recipes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.0 Introduction 1 1.1 Installing Go 1 1.2 Playing Around with Go 3 1.3 Writing a Hello World Program 4 1.4 Using an External Package 5 1.5 Handling Errors 7 1.6 Logging Events 9 1.7 Testing Your Code 10 2. Module Recipes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.0 Introduction 13 2.1 Creating a Go Module 14 2.2 Importing Dependent Packages Into Your Module 15 2.3 Removing Dependent Packages from Your Module 18 2.4 Find Available Versions of Third-Party Packages 19 2.5 Importing a Specific Version of a Dependent Package Into Your Module 20 2.6 Requiring Local Versions of Dependent Packages 22 2.7 Using Multiple Versions of the Same Dependent Packages 26 3. Error Handling Recipes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 3.0 Introduction 29 3.1 Handling Errors 30 3.2 Simplifying Repetitive Error Handling 32 v
📄 Page
8
3.3 Creating Customized Errors 34 3.4 Wrapping an Error with Other Errors 36 3.5 Inspecting Errors 37 3.6 Handling Errors with Panic 39 3.7 Recovering from Panic 41 3.8 Handling Interrupts 43 4. Logging Recipes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 4.0 Introduction 45 4.1 Writing to Logs 45 4.2 Change What Is Being Logged by the Standard Logger 48 4.3 Logging to File 49 4.4 Using Log Levels 50 4.5 Logging to the System Log Service 53 5. Function Recipes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 5.0 Introduction 57 5.1 Defining a Function 57 5.2 Accepting Multiple Data Types with a Function 59 5.3 Accepting a Variable Number of Parameters 61 5.4 Accepting Parameters of Any Type 62 5.5 Creating an Anonymous Function 65 5.6 Creating a Function That Maintains State After It Is Called 66 6. String Recipes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 6.0 Introduction 71 6.1 Creating Strings 71 6.2 Converting String to Bytes and Bytes to String 73 6.3 Creating Strings from Other Strings and Data 73 6.4 Converting Strings to Numbers 77 6.5 Converting Numbers to Strings 79 6.6 Replacing Multiple Characters in a String 81 6.7 Creating a Substring from a String 84 6.8 Checking if a String Contains Another String 85 6.9 Splitting a String Into an Array of Strings or Combining an Array of Strings Into a String 86 6.10 Trimming Strings 88 6.11 Capturing String Input from the Command Line 89 6.12 Escaping and Unescaping HTML Strings 91 6.13 Using Regular Expressions 92 vi | Table of Contents
📄 Page
9
7. General Input/Output Recipes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 7.0 Introduction 97 7.1 Reading from an Input 98 7.2 Writing to an Output 99 7.3 Copying from a Reader to a Writer 100 7.4 Reading from a Text File 102 7.5 Writing to a Text File 104 7.6 Using a Temporary File 106 8. CSV Recipes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 8.0 Introduction 109 8.1 Reading the Whole CSV File 110 8.2 Reading a CSV File One Row at a Time 111 8.3 Unmarshalling CSV Data Into Structs 112 8.4 Removing the Header Line 113 8.5 Using Different Delimiters 113 8.6 Ignoring Rows 114 8.7 Writing CSV Files 115 8.8 Writing to File One Row at a Time 116 9. JSON Recipes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 9.0 Introduction 117 9.1 Parsing JSON Data Byte Arrays to Structs 117 9.2 Parsing Unstructured JSON Data 121 9.3 Parsing JSON Data Streams Into Structs 124 9.4 Creating JSON Data Byte Arrays from Structs 131 9.5 Creating JSON Data Streams from Structs 133 9.6 Omitting Fields in Structs 136 10. Binary Recipes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 10.0 Introduction 139 10.1 Encoding Data to gob Format Data 140 10.2 Decoding gob Format Data to Structs 141 10.3 Encoding Data to a Customized Binary Format 144 10.4 Decoding Data with a Customized Binary Format to Structs 147 11. Date and Time Recipes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 11.0 Introduction 151 11.1 Telling Time 152 11.2 Doing Arithmetic with Time 152 Table of Contents | vii
📄 Page
10
11.3 Representing Dates 153 11.4 Representing Time Zones 154 11.5 Representing Duration 155 11.6 Pausing for a Specific Duration 156 11.7 Measuring Lapsed Time 156 11.8 Formatting Time for Display 159 11.9 Parsing Time Displays Into Structs 163 12. Structs Recipes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167 12.0 Introduction 167 12.1 Defining Structs 168 12.2 Creating Struct Methods 170 12.3 Creating and Using Interfaces 172 12.4 Creating Struct Instances 175 12.5 Creating One-Time Structs 178 12.6 Composing Structs from Other Structs 181 12.7 Defining Metadata for Struct Fields 184 13. Data Structure Recipes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187 13.0 Introduction 187 13.1 Creating Arrays or Slices 188 13.2 Accessing Arrays or Slices 190 13.3 Modifying Arrays or Slices 192 13.4 Making Arrays and Slices Safe for Concurrent Use 195 13.5 Sorting Arrays of Slices 198 13.6 Creating Maps 202 13.7 Accessing Maps 203 13.8 Modifying Maps 204 13.9 Sorting Maps 205 14. More Data Structure Recipes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207 14.0 Introduction 207 14.1 Creating Queues 208 14.2 Creating Stacks 210 14.3 Creating Sets 212 14.4 Creating Linked Lists 216 14.5 Creating Heaps 221 14.6 Creating Graphs 225 14.7 Finding the Shortest Path on a Graph 229 viii | Table of Contents
📄 Page
11
15. Image-Processing Recipes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235 15.0 Introduction 235 15.1 Loading an Image from a File 237 15.2 Saving an Image to a File 238 15.3 Creating Images 239 15.4 Flipping an Image Upside Down 240 15.5 Converting an Image to Grayscale 243 15.6 Resizing an Image 245 16. Networking Recipes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247 16.0 Introduction 247 16.1 Creating a TCP Server 248 16.2 Creating a TCP Client 252 16.3 Creating a UDP Server 254 16.4 Creating a UDP Client 256 17. Web Recipes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259 17.0 Introduction 259 17.1 Creating a Simple Web Application 260 17.2 Handling HTTP Requests 263 17.3 Handling HTML Forms 266 17.4 Uploading a File to a Web Application 268 17.5 Serving Static Files 269 17.6 Creating a JSON Web Service API 274 17.7 Serving Through HTTPS 276 17.8 Using Templates for Go Web Applications 280 17.9 Making an HTTP Client Request 285 18. Testing Recipes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291 18.0 Introduction 291 18.1 Automating Functional Tests 292 18.2 Running Multiple Test Cases 293 18.3 Setting Up and Tearing Down Before and After Tests 295 18.4 Creating Subtests to Have Finer Control Over Groups of Test Cases 297 18.5 Running Tests in Parallel 301 18.6 Generating Random Test Inputs for Tests 306 18.7 Measuring Test Coverage 312 18.8 Testing a Web Application or a Web Service 316 Table of Contents | ix
📄 Page
12
19. Benchmarking Recipes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319 19.0 Introduction 319 19.1 Automating Performance Tests 319 19.2 Running Only Performance Tests 321 19.3 Avoiding Test Fixtures in Performance Tests 322 19.4 Changing the Timing for Running Performance Tests 325 19.5 Running Multiple Performance Test Cases 327 19.6 Comparing Performance Test Results 329 19.7 Profiling a Program 332 Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341 x | Table of Contents
📄 Page
13
Preface Go has been around for more than 10 years. It was publicly announced in 2009, and version 1.0 was released in March 2012. Since 2013 it has gained a steady rise in popularity and is frequently listed among the top 10 most popular programming languages in use today. In the past 10 years there have been plenty of books written about Go, including Go Web Programming, which I wrote in 2015. Most of what needs to be written about Go has already been written; however, the language contin‐ ues to evolve, and there are new generations of would-be Go programmers coming on board. This book came about because of a podcast interview. In September 2021, in the middle of the pandemic, Natalie Pistunovich hosted a “Go Time” podcast interview, titled “Books that Teach Go,” with my friend, Bill Kennedy, and me regarding our Go books. I spoke about Go Web Programming and my new blog site, Go Recipes, which teaches readers how to do the basic stuff with Go. I wanted to provide a steady stream of know-how to serve as a guide for both would-be and experienced Go programmers. After the podcast, Natalie mentioned that, coincidentally, O’Reilly was looking for someone to write a Go cookbook. Since I already had been writing Go recipes, I thought it was too much of a fateful encounter to ignore. Natalie put me in contact with O’Reilly, and the rest became history (and is now part of the Preface)! This cookbook, like many others, is not about teaching new or specific topics but instead explains the basics of common tasks. It covers as much ground as possible on what programmers are most likely to use. The coverage is wide, rather than comprehensive. Each recipe is, more or less, standalone; although at times I cite other recipes, it is not necessary to reference them. You may find some recipes either boring or simple, but there are plenty to choose from! xi
📄 Page
14
Conventions Used in This Book The following typographical conventions are used in this book: Italic Indicates new terms, URLs, email addresses, filenames, and file extensions. Constant width Used for program listings, as well as within paragraphs to refer to program elements such as variable or function names, databases, data types, environment variables, statements, and keywords. Constant width bold Shows commands or other text that should be typed literally by the user. Constant width italic Shows text that should be replaced with user-supplied values or by values deter‐ mined by context. Using Code Examples Supplemental material (code examples, exercises, etc.) is available for download at https://github.com/sausheong/gocookbook. If you have a technical question or a problem using the code examples, please send email to support@oreilly.com. This book is here to help you get your job done. In general, if example code is offered with this book, you may use it in your programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require permission. We appreciate, but generally do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: “Go Cookbook by Sau Sheong Chang (O’Reilly). Copyright 2023 Sau Sheong Chang, 978-1-098-12211-9.” If you feel your use of code examples falls outside fair use or the permission given above, feel free to contact us at permissions@oreilly.com. xii | Preface
📄 Page
15
O’Reilly Online Learning For more than 40 years, O’Reilly Media has provided technol‐ ogy and business training, knowledge, and insight to help companies succeed. Our unique network of experts and innovators share their knowledge and expertise through books, articles, and our online learning platform. O’Reilly’s online learning platform gives you on-demand access to live training courses, in-depth learning paths, interactive coding environments, and a vast collection of text and video from O’Reilly and 200+ other publishers. For more information, visit https://oreilly.com. How to Contact Us Please address comments and questions concerning this book to the publisher: O’Reilly Media, Inc. 1005 Gravenstein Highway North Sebastopol, CA 95472 800-889-8969 (in the United States or Canada) 707-829-7019 (international or local) 707-829-0104 (fax) support@oreilly.com https://www.oreilly.com/about/contact.html We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at https://oreil.ly/go-cookbook. For news and information about our books and courses, visit https://oreilly.com. Find us on LinkedIn: https://linkedin.com/company/oreilly-media. Follow us on Twitter: https://twitter.com/oreillymedia. Watch us on YouTube: https://youtube.com/oreillymedia. Acknowledgments I would like to thank my wife, Angela Lim, and my son, Chang Kai Wen, for bearing with me while I write “just one more book,” for supporting me through this journey, and for their patience over the long weekends and late nights. It can be tough being the family of a writer, but they took it on admirably well (I think). Preface | xiii
📄 Page
16
I want to acknowledge Natalie Pistunovich who not only introduced me to O’Reilly but also helped me review the early chapters of this book. Also, Jon Bodner and Jess Males, both of whom helped me tremendously with their reviews, encouragement, and great suggestions. I also want to thank my current and former colleagues from SP Group, Temasek, and GovTech Singapore for their support and encouragement. The list is too long, but you know who you are and I thank you for your constant support! A writer is nothing without his readers. I appreciate and want to thank all the readers of my blog articles and those who followed my writings on Go as well as various other technologies on my blog site. Your continued support is a great motivator, and I’m very grateful that you find it worthwhile to spend time reading them. Thank you! Finally, I want to thank my late father, Chang Yoon Sang, who passed away last November. He always supported me unconditionally and enthusiastically in every‐ thing I do. He was especially excited that I followed in his footsteps in writing books (he was a traditional Chinese medicine physician and nutritionist, and wrote on both topics, in Chinese). Even though I know he struggled to read my books (he’s not a technical person), I know he was always proud of me, asking me for copies of all my books, including the translations, and wanting to build a shelf full of books his son wrote. My biggest regret is that he never had the chance to add this book to his collection. This book is dedicated to him. xiv | Preface
📄 Page
17
CHAPTER 1 Getting Started Recipes 1.0 Introduction Let’s start easy first. In this chapter, you’ll go through the essential recipes to begin coding in Go, installing Go, and then writing some simple Go code. You will go through the fundamentals, from using external libraries and handling errors to sim‐ ple testing and logging events. Some of these recipes are intentionally concise—more detail about them in later chapters. If you’re already familiar with the basics of Go, you can skip this chapter altogether. 1.1 Installing Go Problem You want to install Go and prepare the environment for Go development. Solution Go to the Go website and download the latest, greatest version of Go. Then follow the installation instructions to install Go. Discussion First, you need to go to the Go download site. You can choose the correct version according to your operating system and hardware and download the right package. 1
📄 Page
18
There are a couple of ways to install Go—you can either install the prebuilt binaries for your platform or compile it from the source. Most of the time, there is no need to compile it from the source unless you can’t find the appropriate prebuilt binaries for your operating system. Even then, you can usually use your operating system’s package manager to install Go instead. MacOS Open the downloaded package file and follow the prompts to install. Go will be installed at /usr/local/go, and your PATH environment variable should also have /usr/ local/go/bin added to it. You can also choose to install using Homebrew, which is usually a version or two behind. To do this, run this from the command line: $ brew update && brew install golang You can set up the PATH later as you like. Linux Extract the downloaded archive file into /usr/local, which should create a go directory. For example, run this from the command line (replace the Go version as necessary): $ rm -rf /usr/local/go && tar -C /usr/local -xzf go1.20.1.linux-amd64.tar.gz You can add /usr/local/go/bin into your PATH as needed by adding this line to your $HOME/.profile: export PATH=$PATH:/usr/local/go/bin The changes will be made the next time you log in, or if you want it to take effect immediately, run source on your profile file: $ source $HOME/.profile Windows Open the downloaded MSI installer file and follow the prompts to install Go. By default, Go will be installed to Program Files or Program Files (x86), but you can always change this. Build from source You shouldn’t build from source unless you cannot find the prebuilt binaries for your operating system. However, if you need to, extract the source files to an appropriate directory first, then run these commands from the command line: $ cd src $ ./all.bash 2 | Chapter 1: Getting Started Recipes
📄 Page
19
If you’re building in Windows, use all.bat instead. The assumption here is that the compilers are already in place. If not, and if you need a deeper dive into installing Go, go to the Installing Go site for details. If all goes well, you should see something like this: ALL TESTS PASSED --- Installed Go for linux/amd64 in /home/you/go. Installed commands in /home/you/go/bin. *** You need to add /home/you/go/bin to your $PATH. *** To check if you have installed Go, you can run the Go tool with the version option to see the installed version: % go version If you have correctly installed Go, you should see something like this: go version go1.20.1 darwin/amd64 1.2 Playing Around with Go Problem You want to write and execute Go code without downloading and installing Go. Solution Use the Go Playground to run your Go code. Discussion The Go Playground runs on Google’s servers, and you can run your program inside a sandbox. Go to the Go Playground URL on your browser. This online environment lets you play with Go code, running the latest Go version (you can switch to a different version). The same web page returns output but only supports standard output and standard error (see Figure 1-1). In a pinch, the Go Playground is a good option to test some Go code. You can also use the Go Playground to share executable code directly. 1.2 Playing Around with Go | 3
📄 Page
20
Figure 1-1. Go Playground 1.3 Writing a Hello World Program Problem You want to create a simple Hello World program in Go. Solution Write the Hello World code in Go, build, and run it. 4 | Chapter 1: Getting Started Recipes