Security with Go (John Daniel Leon) (z-library.sk, 1lib.sk, z-lib.sk)

📄 File Format: PDF

💾 File Size: 2.9 MB

📖 Read Online ⬇️ Download

Registered users can read the full content for free

Register as a Gaohf Library member to read the complete e-book online for free and enjoy a better reading experience.

📄 Page 1

(This page has no text content)

📄 Page 2

Security with Go Explore the power of Golang to secure host, web, and cloud services John Daniel Leon BIRMINGHAM - MUMBAI

📄 Page 3

Security with Go Copyright © 2018 Packt Publishing All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information. Acquisition Editors: Dominic Shakeshaft, Suresh M Jain Project Editor: Alish Firasta Content Development Editor: Monika Sangwan Technical Editors: Joel D'souza, Bhagyashree Rai Copy Editor: Tom Jacob Proofreader: Safis Editing Indexer: Tejal Daruwale Soni Graphics: Tania Dutta Production Coordinator: Aparna Bhagat First published: January 2018 Production reference: 1300118 Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK. ISBN 978-1-78862-791-7 www.packtpub.com

📄 Page 4

mapt.io Mapt is an online digital library that gives you full access to over 5,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website. Why subscribe? Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionals Improve your learning with Skill Plans built especially for you Get a free eBook or video every month Mapt is fully searchable Copy and paste, print, and bookmark content PacktPub.com Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at service@packtpub.com for more details. At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks.

📄 Page 5

Contributors About the author John Daniel Leon is a security expert and developer residing in Houston, TX who currently works at IBM Cloud as the Application Security Architect. John maintains DevDungeon.com, a virtual hackerspace for developers and security experts. He has been programming from a young age and has a B.S. in Computer Science from University of North Texas. He spoke at GopherCon 2016 about packet capturing. John is a polyglot programmer with a strong interest in Python, Go, and Java. Outside of programming and security, he has a background in music theory, performance, and plays violin and guitar.

📄 Page 6

About the reviewer Karthik Gaekwad is a veteran engineer who enjoys building software products from scratch, using cloud and container technologies. He has worked in both large enterprises and startups with his career spanning from National Instruments to Mentor Graphics. He was also the first hired engineer at Signal Sciences, an early engineer at StackEngine (which was acquired by Oracle). Currently, Karthik works at Oracle as a principal engineer to build products in the cloud native space. Karthik has programmed in many languages, including Java, C, C#, Python, and Ruby. He first learned Golang in 2012, and it has been his language of choice ever since. He has written many production-level applications using Go and appreciates its simplicity and flexibility. Karthik graduated from the University of Arizona in 2007 with an MS in Computer Engineering and currently lives in Austin, Texas with his family. Karthik organizes several conferences, including devopsdays and Container Days, and he has chaired the DevOps tracks for the Agile Conference and All Day DevOps. He is also an accomplished author for LinkedIn Learning and Lynda.com. In his free time, Karthik enjoys to spend time with family, keep up with the latest trends in software, and dabble in new product ideas. Packt is searching for authors like you If you're interested in becoming an author for Packt, please visit authors.packtpub.com and apply today. We have worked with thousands of developers and tech professionals, just like you, to help them share their insight with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea.

📄 Page 7

Table of Contents Chapter 1: Introduction to Security with Go 8 About Go 8 Go language design 9 The History of Go 10 Adoption and community 11 Common criticisms about Go 12 The Go toolchain 12 Go mascot 13 Learning Go 14 Why use Go? 14 Why use Go for security? 15 Why not use Python? 15 Why not use Java? 16 Why not use C++? 17 Development environment 17 Installing Go on other platforms 18 Other Linux distributions 18 Windows 19 Mac 19 Setting up Go 19 Creating your workspace 19 Setting up environment variables 20 Editors 21 Creating your first package 21 Writing your first program 22 Running the executable file 22 Building the executable file 22 Installing the executable file 22 Formatting with go fmt 23 Running Go examples 24 Building a single Go file 24 Running a single Go file 24 Building multiple Go files 24 Building a folder (package) 25 Installing a program for use 25

📄 Page 8

Table of Contents [ ii ] Summary 25 Chapter 2: The Go Programming Language 26 Go language specification 27 The Go playground 27 A tour of Go 28 Keywords 29 Notes about source code 31 Comments 31 Types 31 Boolean 31 Numeric 32 Generic numbers 32 Specific numbers 32 Unsigned integers 32 Signed integers 33 Floating point numbers 33 Other numeric types 33 String 34 Array 35 Slice 35 Struct 38 Pointer 39 Function 39 Interface 41 Map 43 Channel 44 Control structures 45 if 46 for 47 range 47 switch, case, fallthrough, and default 48 goto 49 Defer 50 Packages 51 Classes 51 Inheritance 52 Polymorphism 53 Constructors 53 Methods 54

📄 Page 9

Table of Contents [ iii ] Operator overloading 55 Goroutines 55 Getting help and documentation 56 Online Go documentation 56 Offline Go documentation 57 Summary 57 Chapter 3: Working with Files 58 File basics 59 Creating an empty file 59 Truncating a file 60 Getting the file info 61 Renaming a file 61 Deleting a file 62 Opening and closing files 62 Checking whether a file exists 63 Checking read and write permissions 64 Changing permissions, ownership, and timestamps 65 Hard links and symlinks 66 Reading and writing 67 Copying a file 68 Seeking positions in a file 69 Writing bytes to a file 70 Quickly writing to a file 71 Buffered writer 71 Reading up to n bytes from a file 74 Reading exactly n bytes 75 Reading at least n bytes 75 Reading all bytes of a file 76 Quickly reading whole files to memory 77 Buffered reader 78 Reading with a scanner 80 Archives 81 Archive (ZIP) files 81 Extracting (unzip) archived files 83 Compression 84 Compressing a file 85 Uncompressing a File 86 Creating temporary files and directories 87

📄 Page 10

Table of Contents [ iv ] Downloading a file over HTTP 88 Summary 89 Chapter 4: Forensics 90 Files 91 Getting file information 91 Finding the largest files 93 Finding recently modified files 94 Reading the boot sector 96 Steganography 98 Generating an image with random noise 98 Creating a ZIP archive 99 Creating a steganographic image archive 100 Detecting a ZIP archive in a JPEG image 102 Network 103 Looking up a hostname from an IP address 103 Looking up IP addresses from a hostname 104 Looking up MX records 105 Looking up nameservers for a hostname 105 Summary 106 Chapter 5: Packet Capturing and Injection 108 Prerequisites 109 Installing libpcap and Git 109 Installing libpcap on Ubuntu 109 Installing libpcap on Windows 110 Installing libpcap on macOS 110 Installing gopacket 110 Permission problems 110 Getting a list of network devices 111 Capturing packets 112 Capturing with filters 114 Saving to the pcap file 116 Reading from a pcap file 117 Decoding packet layers 119 Creating a custom layer 121 Converting bytes to and from packets 124 Creating and sending packets 125 Decoding packets faster 128 Summary 129

📄 Page 11

Table of Contents [ v ] Chapter 6: Cryptography 131 Hashing 132 Hashing small files 132 Hashing large files 134 Storing passwords securely 135 Encryption 137 Cryptographically secure pseudo-random number generator (CSPRNG) 137 Symmetric encryption 138 AES 138 Asymmetric encryption 143 Generating a public and private key pair 144 Digitally signing a message 147 Verifying a signature 151 TLS 154 Generating a self-signed certificate 154 Creating a certificate signing request 158 Signing a certificate request 161 TLS server 162 TLS client 164 Other encryption packages 167 OpenPGP 167 Off The Record (OTR) messaging 167 Summary 168 Chapter 7: Secure Shell (SSH) 169 Using the Go SSH client 171 Authentication methods 171 Authenticating with a password 172 Authenticating with private key 172 Verifying remote host 174 Executing a command over SSH 177 Starting an interactive shell 179 Summary 182 Chapter 8: Brute Force 183 Brute forcing HTTP basic authentication 185 Brute forcing the HTML login form 188 Brute forcing SSH 191 Brute forcing database login 194 Summary 200 Chapter 9: Web Applications 201 HTTP server 202

📄 Page 12

Table of Contents [ vi ] Simple HTTP servers 203 HTTP basic auth 204 Using HTTPS 205 Creating secure cookies 206 HTML escaping output 208 Middleware with Negroni 208 Logging requests 210 Adding secure HTTP headers 210 Serving static files 212 Other best practices 212 CSRF tokens 213 Preventing user enumeration and abuse 214 Registration 214 Login 214 Resetting the password 214 User profiles 215 Preventing LFI and RFI abuse 215 Contaminated files 216 HTTP client 216 The basic HTTP request 217 Using the client SSL certificate 217 Using a proxy 218 Using system proxy 218 Using a specific HTTP proxy 219 Using a SOCKS5 proxy (Tor) 220 Summary 221 Chapter 10: Web Scraping 223 Web scraping fundamentals 224 Finding strings in HTTP responses with the strings package 225 Using regular expressions to find email addresses in a page 227 Extracting HTTP headers from an HTTP response 230 Setting cookies with an HTTP client 232 Finding HTML comments in a web page 234 Finding unlisted files on a web server 236 Changing the user agent of a request 239 Fingerprinting web application technology stacks 241 Fingerprinting based on HTTP response headers 241 Fingerprinting web applications 243 How to prevent fingerprinting of your applications 245 Using the goquery package for web scraping 246 Listing all hyperlinks in a page 246

📄 Page 13

Table of Contents [ vii ] Finding documents in a web page 247 Listing page title and headings 249 Crawling pages on the site that store the most common words 251 Printing a list of external JavaScript files in a page 252 Depth-first crawling 254 Breadth-first crawling 257 How to protect against web scraping 258 Summary 259 Chapter 11: Host Discovery and Enumeration 260 TCP and UDP sockets 261 Creating a server 261 Creating a client 262 Port scanning 263 Grabbing a banner from a service 264 Creating a TCP proxy 266 Finding named hosts on a network 267 Fuzzing a network service 269 Summary 271 Chapter 12: Social Engineering 273 Gathering intel via JSON REST API 274 Sending phishing emails with SMTP 276 Generating QR codes 280 Base64 encoding data 282 Honeypots 283 TCP honeypot 284 The TCP testing tool 286 HTTP POST form login honeypot 287 HTTP form field honeypots 289 Sandboxing 289 Summary 290 Chapter 13: Post Exploitation 291 Cross compiling 292 Creating bind shells 294 Creating reverse bind shells 296 Creating web shells 297 Finding writable files 299 Changing file timestamp 301 Changing file permissions 302

📄 Page 14

Table of Contents [ viii ] Changing file ownership 303 Summary 305 Chapter 14: Conclusions 307 Recapping the topics you have learned 307 More thoughts on the usage of Go 308 What I hope you take away from the book 308 Be aware of legal, ethical, and technical boundaries 309 Where to go from here 310 Getting help and learning more 310 Another Book You May Enjoy 312 Leave a review – let other readers know what you think 313 Index 314

📄 Page 15

Preface This book covers the Go programming language and explains how to apply it in the cybersecurity industry. The topics covered are useful for red and blue teams, as well as for developers who want to write secure code, and for networking and operations engineers who want to secure their networks, hosts, and intellectual property. The source code examples are all fully functional programs. The examples are intended to be practical applications that you will likely want to incorporate into your own toolkit. Moreover, this book serves as a practical cookbook for building your own custom applications. I have also shared other security best practices and tricks that I learned. This book will walk you through examples of code that are useful in a variety of computer security situations. As you work through the book, you will build a cookbook of practical applications and building blocks to use in your own security tools for use in your organization and engagements. It will also cover some tips and trivia regarding the Go programming language and provide many useful reference programs to boost your own Go cookbook. This book will cover several blue team and red team use cases and various other security- related topics. Blue team topics, namely steganography, forensics, packet capturing, honeypots, and cryptography, and red team topics, namely brute forcing, port scanning, bind and reverse shells, SSH clients, and web scraping, will be covered. Each chapter relates to a different security topic and walks through code examples pertaining to that topic. If you follow this book, you will have a cookbook full of useful security tools and building blocks to create your own custom tools all in Go. This book is not an in-depth tutorial on using the Go language. One chapter is dedicated to explaining Go; however, it only scratches the surface as compared to Alan Donovan and Brian Kernighan's almost 400-page The Go Programming Language. Fortunately, Go is a very easy language to pick up and has a quick learning curve. Some resources on learning Go are provided, but the reader may need to do some supplemental reading if not familiar with Go. This book will not explore cutting-edge security techniques or vulnerabilities that are not already well documented. There are no zero-days or grand techniques unveiled. Each chapter is dedicated to a different security topic. Each one of these topics could have a book written about them. There are experts who specialize in each of these fields, so the book does not go in depth on any particular topic. The reader will have a solid foundation to explore any topic deeper when complete.

📄 Page 16

Preface [ 2 ] Who this book is for This book is for programmers who are already familiar with the Go programming language. Some knowledge of Go is needed, but the reader does not need to be a Go expert. The content is aimed at newcomers to Go, but it will not teach you everything about using Go. Those new to Go will get to explore and experiment with various aspects of Go, while applying it toward security practices. We will start with smaller and simpler examples before moving on to examples that make use of more advanced Go language features. The reader is not expected to be an advanced security expert, but at least should have a basic understanding of core security concepts. The goal is to work through security topics as an experienced developer or security expert who is looking to improve their toolset and grow a library of Go reference code. Readers who like to build cookbooks full of useful tools will enjoy working through these chapters. People who want to build custom tools in Go related to security, networking, and other fields will benefit from the examples. Developers, penetration testers, SOC analysts, DevOps engineers, social engineers, and network engineers can all make use of the contents in this book. What this book covers Chapter 1, Introduction to Security with Go, covers the history of Go and discusses why Go is a good choice for security applications, how to set up a development environment, and run your first program. Chapter 2, The Go Programming Language, presents the basics of programming with Go. It reviews the keywords and data types along with the notable features of Go. It also contains information for getting help and reading documentation. Chapter 3, Working with Files, helps you explore various ways of manipulating, reading, writing, and compressing files with Go. Chapter 4, Forensics, talks about basic file forensics, steganography, and network forensics techniques. Chapter 5, Packet Capturing and Injection, covers various aspects of packet capturing with the gopacket package. Topics include getting a list of network devices, capturing packets from a live network device, filtering packets, decoding packet layers, and sending custom packets. Chapter 6, Cryptography, explains hashing, symmetric encryption such as AES, and asymmetric encryption such as RSA, digital signatures, verifying signatures, TLS connections, generating keys and certificates, and other cryptography packages.

📄 Page 17

Preface [ 3 ] Chapter 7, Secure Shell (SSH), covers the Go SSH package, how to use the client to authenticate with a password and with a key pair. It also covers how to execute commands on a remote host using SSH and running an interactive shell. Chapter 8, Brute Force, includes examples of multiple brute force attack clients including HTTP basic authentication, HTML login form, SSH, MongoDB, MySQL, and PostgreSQL. Chapter 9, Web Applications, explains how to build secure web applications with secure cookies, sanitized output, security headers, logging, and other best practices. It also covers writing secure web clients that utilize client certificates, HTTP proxies, and SOCKS5 proxies such as Tor. Chapter 10, Web Scraping, discusses basic scraping techniques such as string matching, regular expressions, and fingerprinting. It also covers the goquery package, a powerful tool for extracting data from structured web pages. Chapter 11, Host Discovery and Enumeration, covers port scanning, banner grabbing, TCP proxies, simple socket server and client, fuzzing, and scanning networks for named hosts. Chapter 12, Social Engineering, provides examples for gathering intel via a JSON REST API such as Reddit, sending phishing emails with SMTP, and generating QR codes. It also covers Honeypots along with TCP and HTTP honeypot examples. Chapter 13, Post Exploitation, covers various post exploitation techniques such as cross- compiling bind shells, reverse bind shells, and web shells. It also provides examples of searching for writable files and modifying timestamp, ownership, and permissions. Chapter 14, Conclusions, is a recap of topics, showing you where you can go from here, and also has considerations for applying the techniques learned in this book. To get the most out of this book Readers should have basic programming knowledge and understanding of at1. least one programming language. To run the examples the reader needs a computer with Go installed. Installation2. instructions are covered in the book. The recommended operating system is Ubuntu Linux, but examples should also run on macOS, Windows, and other Linux distributions.

📄 Page 18

Preface [ 4 ] Download the example code files You can download the example code files for this book from your account at www.packtpub.com. If you purchased this book elsewhere, you can visit www.packtpub.com/support and register to have the files emailed directly to you. You can download the code files by following these steps: Log in or register at www.packtpub.com.1. Select the SUPPORT tab.2. Click on Code Downloads & Errata.3. Enter the name of the book in the Search box and follow the onscreen4. instructions. Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of one of these: WinRAR/7-Zip for Windows Zipeg/iZip/UnRarX for Mac 7-Zip/PeaZip for Linux The code bundle for the book is also hosted on GitHub at https:/ ​/ ​github. ​com/ PacktPublishing/​Security- ​with- ​Go. We also have other code bundles from our rich catalog of books and videos available at https:/ ​/​github. ​com/ ​PacktPublishing/ ​. Check them out! Conventions used There are a number of text conventions used throughout this book. CodeInText: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: " The make() function will create a slice of a certain type with a certain length and capacity."

📄 Page 19

Preface [ 5 ] A block of code is set as follows: package main import ( "fmt" ) func main() { // Basic for loop for i := 0; i < 3; i++ { fmt.Println("i:", i) } // For used as a while loop n := 5 for n < 10 { fmt.Println(n) n++ } } When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold: package main import ( "fmt" ) func main() { // Basic for loop for i := 0; i < 3; i++ { fmt.Println("i:", i) } // For used as a while loop n := 5 for n < 10 { fmt.Println(n) n++ } }

📄 Page 20

Preface [ 6 ] Any command-line input or output is written as follows: sudo apt-get install golang-go Bold: Indicates a new term, an important word, or words that you see onscreen. For example, words in menus or dialog boxes appear in the text like this. Here is an example: " In Windows 10, this can be found by navigating to Control Panel | System | Advanced System Settings | Environment Variables." Warnings or important notes appear like this. Tips and tricks appear like this. Get in touch Feedback from our readers is always welcome. General feedback: Email feedback@packtpub.com and mention the book title in the subject of your message. If you have questions about any aspect of this book, please email us at questions@packtpub.com. Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details. Piracy: If you come across any illegal copies of our works in any form on the Internet, we would be grateful if you would provide us with the location address or website name. Please contact us at copyright@packtpub.com with a link to the material. If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.