100 Go Mistakes and How to Avoid Them (Teiva Harsanyi) (Z-Library)

M A N N I N G Teiva Harsanyi

2 CHAPTER

100 Go Mistakes and How to Avoid Them

100 Go Mistakes AND HOW TO AVOID THEM TEIVA HARSANYI M A N N I N G SHELTER ISLAND

For online information and ordering of this and other Manning books, please visit www.manning.com. The publisher offers discounts on this book when ordered in quantity. For more information, please contact Special Sales Department Manning Publications Co. 20 Baldwin Road PO Box 761 Shelter Island, NY 11964 Email: orders@manning.com ©2022 by Manning Publications Co. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by means electronic, mechanical, photocopying, or otherwise, without prior written permission of the publisher. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in the book, and Manning Publications was aware of a trademark claim, the designations have been printed in initial caps or all caps. Recognizing the importance of preserving what has been written, it is Manning’s policy to have the books we publish printed on acid-free paper, and we exert our best efforts to that end. Recognizing also our responsibility to conserve the resources of our planet, Manning books are printed on paper that is at least 15 percent recycled and processed without the use of elemental chlorine. The author and publisher have made every effort to ensure that the information in this book was correct at press time. The author and publisher do not assume and hereby disclaim any liability to any party for any loss, damage, or disruption caused by errors or omissions, whether such errors or omissions result from negligence, accident, or any other cause, or from any usage of the information herein. Manning Publications Co. Development editor: Doug Rudder 20 Baldwin Road Technical development editor: Arthur Zubarev PO Box 761 Review editor: Mihaela Batinić Shelter Island, NY 11964 Production editor: Deirdre S. Hiam Copy editors: Frances Buran and Tiffany Taylor Proofreader: Katie Tennant Technical proofreader: Tim van Deurzen Typesetter and cover designer: Marija Tudor ISBN 9781617299599 Printed in the United States of America

To Davy Harsanyi: Keep being the person you are, little brother; the stars are your limit. À Mélissa, ma puce.

contents preface xii acknowledgments xiii about this book xiv about the author xvii about the cover illustration xviii 1 Go: Simple to learn but hard to master 1 1.1 Go outline 2 1.2 Simple doesn’t mean easy 3 1.3 100 Go mistakes 4 Bugs 4 ■ Needless complexity 5 ■ Weaker readability 5 Suboptimal or unidiomatic organization 5 ■ Lack of API convenience 6 ■ Under-optimized code 6 ■ Lack of productivity 6 2 Code and project organization 7 2.1 #1: Unintended variable shadowing 7 2.2 #2: Unnecessary nested code 9 2.3 #3: Misusing init functions 12 Concepts 12 ■ When to use init functions 15 2.4 #4: Overusing getters and setters 17 2.5 #5: Interface pollution 18 Concepts 18 ■ When to use interfaces 21 ■ Interface pollution 24vi

CONTENTS vii2.6 #6: Interface on the producer side 25 2.7 #7: Returning interfaces 27 2.8 #8: any says nothing 28 2.9 #9: Being confused about when to use generics 31 Concepts 31 ■ Common uses and misuses 34 2.10 #10: Not being aware of the possible problems with type embedding 36 2.11 #11: Not using the functional options pattern 40 Config struct 41 ■ Builder pattern 42 ■ Functional options pattern 44 2.12 #12: Project misorganization 46 Project structure 46 ■ Package organization 47 2.13 #13: Creating utility packages 48 2.14 #14: Ignoring package name collisions 50 2.15 #15: Missing code documentation 51 2.16 #16: Not using linters 53 3 Data types 56 3.1 #17: Creating confusion with octal literals 56 3.2 #18: Neglecting integer overflows 57 Concepts 58 ■ Detecting integer overflow when incrementing 59 Detecting integer overflows during addition 60 Detecting an integer overflow during multiplication 60 3.3 #19: Not understanding floating points 61 3.4 #20: Not understanding slice length and capacity 64 3.5 #21: Inefficient slice initialization 68 3.6 #22: Being confused about nil vs. empty slices 71 3.7 #23: Not properly checking if a slice is empty 74 3.8 #24: Not making slice copies correctly 76 3.9 #25: Unexpected side effects using slice append 77 3.10 #26: Slices and memory leaks 80 Leaking capacity 80 ■ Slice and pointers 81 3.11 #27: Inefficient map initialization 84 Concepts 84 ■ Initialization 86

CONTENTSviii3.12 #28: Maps and memory leaks 87 3.13 #29: Comparing values incorrectly 90 4 Control structures 95 4.1 #30: Ignoring the fact that elements are copied in range loops 95 Concepts 96 ■ Value copy 96 4.2 #31: Ignoring how arguments are evaluated in range loops 98 Channels 100 ■ Array 100 4.3 #32: Ignoring the impact of using pointer elements in range loops 102 4.4 #33: Making wrong assumptions during map iterations 105 Ordering 105 ■ Map insert during iteration 107 4.5 #34: Ignoring how the break statement works 108 4.6 #35: Using defer inside a loop 110 5 Strings 113 5.1 #36: Not understanding the concept of a rune 114 5.2 #37: Inaccurate string iteration 115 5.3 #38: Misusing trim functions 118 5.4 #39: Under-optimized string concatenation 119 5.5 #40: Useless string conversions 121 5.6 #41: Substrings and memory leaks 123 6 Functions and methods 126 6.1 #42: Not knowing which type of receiver to use 127 6.2 #43: Never using named result parameters 129 6.3 #44: Unintended side effects with named result parameters 132 6.4 #45: Returning a nil receiver 133 6.5 #46: Using a filename as a function input 136 6.6 #47: Ignoring how defer arguments and receivers are evaluated 138 Argument evaluation 138 ■ Pointer and value receivers 141

CONTENTS ix7 Error management 143 7.1 #48: Panicking 143 7.2 #49: Ignoring when to wrap an error 146 7.3 #50: Checking an error type inaccurately 149 7.4 #51: Checking an error value inaccurately 152 7.5 #52: Handling an error twice 154 7.6 #53: Not handling an error 156 7.7 #54: Not handling defer errors 158 8 Concurrency: Foundations 162 8.1 #55: Mixing up concurrency and parallelism 163 8.2 #56: Thinking concurrency is always faster 166 Go scheduling 166 ■ Parallel merge sort 169 8.3 #57: Being puzzled about when to use channels or mutexes 173 8.4 #58: Not understanding race problems 174 Data races vs. race conditions 174 ■ The Go memory model 179 8.5 #59: Not understanding the concurrency impacts of a workload type 181 8.6 #60: Misunderstanding Go contexts 186 Deadline 186 ■ Cancellation signals 187 ■ Context values 188 ■ Catching a context cancellation 190 9 Concurrency: Practice 193 9.1 #61: Propagating an inappropriate context 193 9.2 #62: Starting a goroutine without knowing when to stop it 196 9.3 #63: Not being careful with goroutines and loop variables 198 9.4 #64: Expecting deterministic behavior using select and channels 200 9.5 #65: Not using notification channels 204 9.6 #66: Not using nil channels 205 9.7 #67: Being puzzled about channel size 211 9.8 #68: Forgetting about possible side effects with string formatting 213 etcd data race 213 ■ Deadlock 214

CONTENTSx9.9 #69: Creating data races with append 216 9.10 #70: Using mutexes inaccurately with slices and maps 218 9.11 #71: Misusing sync.WaitGroup 221 9.12 #72: Forgetting about sync.Cond 223 9.13 #73: Not using errgroup 228 9.14 #74: Copying a sync type 231 10 The standard library 234 10.1 #75: Providing a wrong time duration 235 10.2 #76: time.After and memory leaks 235 10.3 #77: Common JSON-handling mistakes 238 Unexpected behavior due to type embedding 238 ■ JSON and the monotonic clock 240 ■ Map of any 243 10.4 #78: Common SQL mistakes 244 Forgetting that sql.Open doesn’t necessarily establish connections to a database 244 ■ Forgetting about connections pooling 245 Not using prepared statements 246 ■ Mishandling null values 247 ■ Not handling row iteration errors 248 10.5 #79: Not closing transient resources 249 HTTP body 250 ■ sql.Rows 252 ■ os.File 253 10.6 #80: Forgetting the return statement after replying to an HTTP request 255 10.7 #81: Using the default HTTP client and server 256 HTTP client 256 ■ HTTP server 259 11 Testing 262 11.1 #82: Not categorizing tests 262 Build tags 263 ■ Environment variables 264 Short mode 265 11.2 #83: Not enabling the -race flag 266 11.3 #84: Not using test execution modes 268 The parallel flag 269 ■ The -shuffle flag 270 11.4 #85: Not using table-driven tests 271 11.5 #86: Sleeping in unit tests 274 11.6 #87: Not dealing with the time API efficiently 278

CONTENTS xi11.7 #88: Not using testing utility packages 281 The httptest package 281 ■ The iotest package 283 11.8 #89: Writing inaccurate benchmarks 285 Not resetting or pausing the timer 286 ■ Making wrong assumptions about micro-benchmarks 287 ■ Not being careful about compiler optimizations 289 ■ Being fooled by the observer effect 291 11.9 #90: Not exploring all the Go testing features 294 Code coverage 294 ■ Testing from a different package 295 Utility functions 295 ■ Setup and teardown 296 12 Optimizations 299 12.1 #91: Not understanding CPU caches 300 CPU architecture 300 ■ Cache line 301 ■ Slice of structs vs. struct of slices 304 ■ Predictability 305 ■ Cache placement policy 307 12.2 #92: Writing concurrent code that leads to false sharing 312 12.3 #93: Not taking into account instruction-level parallelism 315 12.4 #94: Not being aware of data alignment 321 12.5 #95: Not understanding stack vs. heap 324 Stack vs. heap 324 ■ Escape analysis 329 12.6 #96: Not knowing how to reduce allocations 331 API changes 331 ■ Compiler optimizations 332 ■ sync.Pool 332 12.7 #97: Not relying on inlining 334 12.8 #98: Not using Go diagnostics tooling 337 Profiling 337 ■ Execution tracer 344 12.9 #99: Not understanding how the GC works 347 Concepts 347 ■ Examples 349 12.10 #100: Not understanding the impacts of running Go in Docker and Kubernetes 352 Final words 355 index 357

preface In 2019, I started my second professional experience with Go as the primary language. While working in this new context, I noticed some common patterns regarding Go coding mistakes. I started to think that perhaps writing about these frequent mistakes could help some developers. So, I wrote a blog post called “The Top 10 Most Common Mistakes I’ve Seen in Go Projects.” The post was very popular: it had more than 100,000 reads and was selected by the Golang Weekly newsletter as one of the top articles of 2019. Beyond that, I was pleased with the positive feedback I got from the Go community. From that moment, I realized that communicating about common mistakes was a powerful tool. Accompanied by concrete examples, it can help people learn new skills efficiently and facilitate remembering the context of a mistake and how to avoid it. I spent about a year compiling mistakes from various sources such as other profes- sional projects, open source repositories, books, blogs, studies, and discussions with the Go community. To be transparent, I was also a decent source of inspiration regard- ing mistakes. At the end of 2020, I reached 100 Go mistakes, which seemed to me like the right moment to propose my idea to a publisher. I contacted only one: Manning. I saw Man- ning as a top-level company known for publishing high-quality books, and to me, it was the perfect partner. It took me almost 2 years and countless iterations to frame each of the 100 mistakes alongside meaningful examples and multiple solutions where context is key. I hope this book will help you avoid making these common mistakes and help you enhance your proficiency in the Go language.xii

acknowledgments I want to thank a number of people. My parents, for pushing me when I was in a com- plete failure situation during my studies. My uncle Jean-Paul Demont, for helping me find the light. Pierre Gautier, for being a great source of inspiration and making me trust myself. Damien Chambon, for steadily setting the bar higher and pushing me to get better. Laurent Bernard, for being a role model and teaching me that soft skills and communication are crucial. Valentin Deleplace, for the consistency of his excep- tional feedback. Doug Rudder, for teaching me the delicate art of conveying ideas in a written form. Tiffany Taylor and Katie Tennant, for the high-quality copy editing and proofreading, and Tim van Deurzen, for the depth and the quality of his technical review. I also want to thank, Clara Chambon, my beloved little goddaughter; Virginie Chambon, the nicest person alive; the whole Harsanyi family; Afroditi Katika, my favorite PO; Sergio Garcez and Kasper Bentsen, two amazing engineers; and the entire Go community. Lastly, I would like to thank the reviewers: Adam Wanadamaiken, Alessandro Cam- peis, Allen Gooch, Andres Sacco, Anupam Sengupta, Borko Djurkovic, Brad Hor- rocks, Camal Cakar, Charles M. Shelton, Chris Allan, Clifford Thurber, Cosimo Damiano Prete, David Cronkite, David Jacobs, David Moravec, Francis Setash, Gianlu- igi Spagnuolo, Giuseppe Maxia, Hiroyuki Musha, James Bishop, Jerome Meyer, Joel Holmes, Jonathan R. Choate, Jort Rodenburg, Keith Kim, Kevin Liao, Lev Veyde, Mar- tin Dehnert, Matt Welke, Neeraj Shah, Oscar Utbult, Peiti Li, Philipp Janertq, Robert Wenner, Ryan Burrowsq, Ryan Huber, Sanket Naik, Satadru Roy, Shon D. Vick, Thad Meyer, and Vadim Turkov. Your suggestions helped make this a better book.xiii

about this book 100 Go Mistakes and How to Avoid Them contains 100 common mistakes made by Go developers when working with various aspects of the language. It focuses heavily on the core language and the standard library, not external libraries or frameworks. The discussions of most of the mistakes are accompanied by concrete examples to illus- trate when we are likely to make such errors. It’s not a dogmatic book: each solution is detailed to convey the context in which it should apply. Who should read this book This book is for developers with existing knowledge of the Go language. It doesn’t review basic concepts such as syntax or keywords. Ideally, you have already worked on an existing Go project at work or home. But before delving into most topics, we make sure the foundations are clear. How this book is organized: A roadmap 100 Go Mistakes and How to Avoid Them consists of 12 chapters:  Chapter 1, “Go: Simple to learn but hard to master,” describes why despite being considered a simple language, Go isn’t easy to master. It also shows the different types of mistakes we cover in the book.  Chapter 2, “Code and project organization,” contains common mistakes that can prevent us from organizing a codebase in a clean, idiomatic, and maintain- able manner.  Chapter 3, “Data types,” discusses mistakes related to basic types, slices, and maps.xiv

ABOUT THIS BOOK xv Chapter 4, “Control structures,” explores common mistakes related to loops and other control structures.  Chapter 5, “Strings,” looks at the principle of string representation and com- mon mistakes leading to code inaccuracy or inefficiency.  Chapter 6, “Functions and methods,” explores common problems related to functions and methods, such as choosing a receiver type and preventing com- mon defer bugs.  Chapter 7, “Error management,” walks through idiomatic and accurate error handling in Go.  Chapter 8, “Concurrency: Foundations,” presents the fundamental concepts behind concurrency. We discuss topics such as why concurrency isn’t always faster, the differences between concurrency and parallelism, and workload types.  Chapter 9, “Concurrency: Practice,” looks at concrete examples of mistakes related to applying concurrency when using Go channels, goroutines, and other primitives.  Chapter 10, “The standard library,” contains common mistakes made when using the standard library with HTTP, JSON, or (for example) the time API.  Chapter 11, “Testing,” discusses mistakes that make testing and benchmarking more brittle, less effective, and less accurate.  Chapter 12, “Optimizations,” closes the book by exploring how to optimize an application for performance, from understanding CPU fundamentals to Go- specific topics. About the code This book contains many examples of source code both in numbered listings and in line with normal text. In both cases, source code is formatted in a fixed-width font like this to separate it from ordinary text. Sometimes code is also in bold to high- light code that has changed from previous steps in the chapter, such as when a new feature adds to an existing line of code. In many cases, the original source code has been reformatted; we’ve added line breaks and reworked indentation to accommodate the available page space in the book. In some cases, even this was not enough, and listings include line-continuation markers (➥). Additionally, comments in the source code have often been removed from the listings when the code is described in the text. Code annotations accompany many of the listings, highlighting important concepts. You can get executable snippets of code from the liveBook (online) version of this book at https://livebook.manning.com/book/100-go-mistakes-how-to-avoid-them. The complete code for the examples in the book is available for download from the Manning website at https://www.manning.com/books/100-go-mistakes-how-to-avoid -them, and from GitHub at https://github.com/teivah/100-go-mistakes.

ABOUT THIS BOOKxviliveBook discussion forum Purchase of 100 Go Mistakes and How to Avoid Them includes free access to liveBook, Manning’s online reading platform. Using liveBook’s exclusive discussion features, you can attach comments to the book globally or to specific sections or paragraphs. It’s a snap to make notes for yourself, ask and answer technical questions, and receive help from the author and other users. To access the forum, go to https://livebook .manning.com/book/100-go-mistakes-how-to-avoid-them/discussion. You can also learn more about Manning’s forums and the rules of conduct at https://livebook .manning.com/discussion. Manning’s commitment to our readers is to provide a venue where a meaningful dialogue between individual readers and between readers and the author can take place. It is not a commitment to any specific amount of participation on the part of the author, whose contribution to the forum remains voluntary (and unpaid). We sug- gest you try asking the author some challenging questions lest his interest stray! The forum and the archives of previous discussions will be accessible from the publisher’s website as long as the book is in print.

about the author TEIVA HARSANYI is a senior software engineer at Docker. He has worked in various domains, including insurance, transportation, and safety-critical industries like air traffic management. He is very passionate about Go and how to design and imple- ment reliable applications.xvii

about the cover illustration The figure on the cover of 100 Go Mistakes and How to Avoid Them is “Femme de Buc- cari en Croatie,” or “A woman from Bakar, Croatia,” taken from a collection by Jacques Grasset de Saint-Sauveur, published in 1797. Each illustration is finely drawn and colored by hand. In those days, it was easy to identify where people lived and what their trade or sta- tion in life was just by their dress. Manning celebrates the inventiveness and initiative of the computer business with book covers based on the rich diversity of regional cul- ture centuries ago, brought back to life by pictures from collections such as this one. xviii