Go Programming for Network Operations A Golang Network Automation Handbook (Tom McAllen) (z-library.sk, 1lib.sk, z-lib.sk)

Table of Contents About the Author Introduction Why Go for Network Operations? Chapter 1: File and Directory Basics 1.1 Open/Close 1.2 Add/Remove 1.3 Get Info 1.4 List Directory Contents 1.5 File Tree Walk Chapter 2: Reading and Writing File Types 2.1 Plain Text 2.1.1 Reading Plain Text 2.1.2 Writing Plain Text 2.2 CSV 2.2.1 Reading CSV 2.2.2 Writing CSV 2.3 YAML 2.3.1 Reading YAML 2.3.2 Writing YAML 2.4 JSON 2.4.1 Reading JSON 2.4.2 Writing JSON 2.5 XML 2.5.1 Reading XML 2.5.2 Writing XML Chapter 3: Text Templates 3.1 Template Elements 3.2 Basic Template 3.3 Template Inheritance Chapter 4: HTTP Client 4.1 GET 4.2 POST Chapter 5: SSH Client

5.1 Single Command with Private Key Authentication 5.2 Multiple Commands with Password Authentication 5.3 SFTP File Copy with Password Authentication Chapter 6: IP Address Manipulation 6.1 Network Contains IP 6.2 List Network IPs 6.3 Mask Conversion Chapter 7: DNS Record Lookup 7.1 Forward (A) 7.2 Reverse (PTR) 7.3 Canonical Name (CNAME) 7.4 Mail Exchanger (MX) 7.5 Name Servers (NS) 7.6 Service (SRV) 7.7 Text (TXT) Chapter 8: Regex Pattern Matching 8.1 Simple Matching 8.2 Match Groups 8.3 Named Capture Groups 8.4 Template Expansion 8.5 Multi-line Delimited Split Chapter 9: External Commands 9.1 Combined Stdout/Stderr 9.2 Separate Stdout/Stderr Chapter 10: CLI Application Helpers 10.1 User Input 10.2 Command-line Arguments 10.3 Command-line Flags 10.4 Column Formatting

Go Programming for Network Operations A Golang Network Automation Handbook Tom McAllen, CCIE 26867 About the Author Tom McAllen is a developer and network expert with over two decades of experience in programming and network operations. He began his career as a Linux Admin and Web Application Developer and after an internship at Merit Network and the North American Network Operators Group (NANOG) he went on to complete a Bachelors in Computer Science and Masters in Technology Management. Tom also holds over a dozen active technical certifications that include Cisco Certified Internetwork Expert (CCIE) and Certified Information Systems Security Professional (CISSP). Today he works in Austin, TX at a Fortune Global 500 enterprise as a Senior Principal Network Engineer. Disclaimer The information provided in this book is for educational purposes only. The reader is responsible for his or her own actions. While all attempts have been made to verify the information provided in this book, the author does not assume any responsibility for errors, omissions, or contrary interpretations of the subject matter contained within. The author does not accept any responsibility for any liabilities or damages, real or perceived, resulting from the use of this information.

All trademarks and brands within this book are for clarifying purposes only, are owned by the owners themselves, and are not affiliated with this document. The trademarks used are without any consent, and the publication of the trademark is without permission or backing by the trademark owner. © 2018 Tom McAllen. All Rights Reserved. Introduction Go is the most exciting new mainstream programming language to appear in recent decades. Like modern network operations, Go has taken aim at 21st century infrastructures, particularly in the cloud. This book illustrates how to apply Go programming to network operations. The topics cover common use cases through examples that are designed to act as a guide and serve as a reference. The reader is assumed to have already gained a fundamental understanding of Go programming; however, the examples are explained for additional clarification. The focus is on using Go for network operations, not on the language itself. This book's approach is practical; readers are encouraged to experiment with the examples and piece together their own programs to gain hands-on experience. The examples in this book are designed for e-book format. They are short and concise, while maintaining enough content to run and illustrate the point. While error handling is omitted for brevity, it is always recommended to include it, along with other best practices, when developing production code. Why Go for Network Operations? The Go programming language is a modern language released in 2009 by an internal team at Google. As an engineered language (rather than something that evolved over time), Go comes with several key advantages. It uses a

clean and easy-to-understand syntax that avoids the complexity of other languages, such as C++. It is statically typed, yet efficient by supporting short variable declaration, interfaces, and structural typing. Another key advantage is built-in concurrency at the core of the language. The Go concurrency model is simple and effective by leveraging goroutines and channels, rather than less efficient threads. Concurrency is native to the language and scales efficiently as the number of CPU cores increase. Go is lightweight and fast. It is directly compiled to binary rather than depending on an interpreter. This unlocks the performance of running directly on the underlying hardware, while reducing the overhead through its inherent garbage collection. Compilation is fast; therefore, it is practical to use in situations where interpreted languages are normally used. It conveniently compiles to a single binary file and can cross-compile for several operating systems and architectures, including MacOS, Linux, Windows, and BSD. The rise in popularity and adoption in Go clearly demonstrates its benefit in the real world of IT. The next-generation features that are engineered into the language from the beginning can increase confidence when relying on it for production use. As networks continue to evolve and companies look more at programmability and automation to address new challenges, network operations must remain agile using a modern set of tools.

Chapter 1: File and Directory Basics Files and directories are a fundamental feature of most operating systems. The ability to manipulate and interpret them is an essential skill. This chapter focuses on several examples that demonstrate basic file and directory operations. First, this section looks at the fundamentals of opening and closing, and adding and removing files. Then, it covers getting attributes of files, such as size, permissions, ownership and modification date. Finally, the last two sections demonstrate listing files and directories, both in the same directory and recursively. This chapter will cover the following file and directory topics: Opening and closing Adding and removing Getting information Listing directory contents Walking a file and directory tree

Chapter 1.1: Open/Close There are two main options for opening files, the os.Open() function, that simply accepts a filename and provides a ready-only file, and a more enhanced option, the os.OpenFile() function , which enables more control by implementing the flags depicted in the table below. These flags can be combined by separating them with the logical OR pipe operator. Flag Description O_RDONLY Open the file read-only. O_WRONLY Open the file write-only. O_RDWR Open the file read-write. O_APPEND Append data to the file when writing. O_CREATE Create a new file if none exists. O_EXCL Used with O_CREATE. The file must not exist. O_SYNC Open for Synchronous I/O. O_TRUNC If possible, truncate the file when opened. The example below demonstrates the more enhanced option by using the os.OpenFile() function to return a pointer to a value of type os.File. The flag combination creates a write-only file if none exists, otherwise, it appends to the file. The os.File.Write() method is then called on the os.File object to append a byte slice to the file. Finally, the os.File.Close() method is called on the os.File object to render the I/O unusable. package main import "os" func main() {     file, _ := os.OpenFile("file.txt",         os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)

    file.Write([]byte("append data\n"))     file.Close() }

Chapter 1.2: Add/Remove This next example demonstrates adding a directory and file, then removing them. The example starts out by declaring a directory path and file name. Then, leveraging the directory path and setting permission bits, the os.MkdirAll() function is used to create the directory named path. Once the directory is created, the full directory path is built by calling the filepath.Join() function together with the path name. This full path is then used as a parameter in the os.Create() function to create the file and return an os.File object. At this point, the os.File.Close() method is called to close the file and then remove it with the os.Remove() function. Finally, the directory is deleted using the os.RemoveAll() function, which removes the path and any dependent children. package main import (     "os"     "path/filepath" ) func main() {     dpath := "./dir"     fname := "file.txt"     _ = os.MkdirAll(dpath, 0777)     fpath := filepath.Join(dpath, fname)     file, _ := os.Create(fpath)     file.Close()     _ = os.Remove(fpath)

    _ = os.RemoveAll(dpath) }

Chapter 1.3: Get Info Getting information from files and directories is straightforward using the os package. This example leverages the os.Stat() function to return an os.FileInfo object that is then used to read various attributes that include file name, size, permissions, last modified timestamp, and whether it is a directory or not. The os.FileInfo object is then used on the respective methods to retrieve the file and directory metadata. package main import (     "fmt"     "os" ) func main() {     file, _ := os.Stat("main.go")     fl := fmt.Println     fl("File name: ", file.Name())     fl("Size in bytes: ", file.Size())     fl("Last modified: ", file.ModTime())     fl("Is a directory: ", file.IsDir())     ff := fmt.Printf     ff("Permission 9-bit: %s\n", file.Mode())     ff("Permission 3-digit: %o\n", file.Mode())     ff("Permission 4-digit: %04o\n",file.Mode()) } There are three different permission representations in the output below that take their form in the standard Unix 9-bit format, 3-digit octal, and 4-digit octal format. These permission bits are formatted using the fmt package string and base integer verbs. Also, take note that the os.FileInfo.ModTime()

method returns a time.Time object which is ideal for additional parsing and formatting. File name: main.go Size in bytes: 0 Last modified: 2018-10-20 10:51:12.305 -0500 EDT Is a directory: false Permission bits: -rwxr--r-- Permission 3-digit octal: 744 Permission 4-digit octal: 0744

Chapter 1.4: List Directory Contents Listing directory contents within a specific directory can be implemented using one of the I/O utility functions from the ioutil package. By specifying the directory name in the ioutil.ReadDir() function, it conveniently returns a list of os.FileInfo objects, as seen in the previous section. In this example, a period symbol is used to represent the current working directory as an input to the ioutil.ReadDir() function. Then, a loop is used over a list that returns os.FileInfo objects. Finally, the loop iterates through and prints the name and the last modified timestamp by leveraging each of the os.FileInfo.Name() and os.FileInfo.ModTime() methods. package main import (     "fmt"     "io/ioutil" ) func main() {     files, _ := ioutil.ReadDir(".")     for _, file := range files {         fmt.Println(file.Name(), file.ModTime())     } } Notice that the output below is intentionally sorted by file name. This is an inherent feature of the ioutil.ReadDir() function. file1.txt 2018-10-20 14:21:11.1 -0500 EDT file2.txt 2018-10-20 14:22:22.1 -0500 EDT

main.go 2018-10-20 14:27.33.1 -0500 EDT

Chapter 1.5: File Tree Walk Recursively walking a file tree from a designated starting point can be used to find and collect data or modify specific files in a directory structure. The filepath.Walk() function, from the filepath package enables this functionality. It accepts a root directory and a function to process the files and directories; symbolic links are not followed. To demonstrate a file tree walk, the example below begins by setting up a function as a value. Then, it uses the os.FileInfo object on the os.FileInfo.IsDir() method to determine whether the entry is a directory and then prints the result, along with the respective path. Finally, the function is set as a parameter along with the root directory into the filepath.Walk() function. During execution, the initial scan function is called for each file or directory visited by the filepath.Walk() function. package main import (     "fmt"     "os"     "path/filepath" ) func main() {     scan := func(         path string, i os.FileInfo, _ error) error {         fmt.Println(i.IsDir(), path)         return nil     }     _ = filepath.Walk(".", scan) }

The output below is intentionally listed in lexical order. This sorting is an inherent feature of the filepath.Walk() function. false main.go true dir1 false dir1/file1.txt false dir1/file2.txt

Chapter 2: Reading and Writing File Types Reading and writing different file types is a critical ingredient in many programs. This chapter walks through examples that read and write the common file types found in most environments; these include Plain Text, CSV, YAML, JSON, and XML. Several examples will take advantage of a buffer to optimize read or write operations and minimize expensive syscalls. The buffer will be used to store data until a certain size is reached before writing and retrieve more data in a single operation when reading. This chapter will cover the following topics: Plain text CSV YAML JSON XML

Chapter 2.1: Plain Text Reading and writing plain text is typically done line-by-line or word-by- word. The examples in the following subchapters will demonstrate reading and writing lines of text using a native buffer. Leveraging a buffer can be helpful when working with large files since it stores data until a certain size is reached before committing it to IO; this reduces the number of expensive system calls and other mechanics involved. This subchapter will cover the following topics: Reading plain text files Writing plain text files

Chapter 2.1.1: Reading Plain Text The bufio package Scanner is a convenient interface for reading text by lines or words from a file. The native buffer optimizes read operations by retrieving more data in a single operation. This example demonstrates reading text line-by-line from the plain text file below. RTR1 1.1.1.1 RTR2 2.2.2.2 RTR3 3.3.3.3 To begin, the os.Open() function is used to open the above plain text file in read-only mode and then return a pointer to a value of type os.File. Next, the bufio.NewScanner() function is called with the os.File object as its parameter which returns a bufio.Scanner object that is used on the bufio.Scanner.Split() method. Since the goal is to read each line of text, bufio.ScanLines is leveraged as an input to the bufio.Scanner.Split() method and then the Scanner advances to each new line using the bufio.Scanner.Scan() method. As each new line is advanced, each iteration is accessed from the bufio.Scanner.Text() method which is then appended to a slice and printed at the end. Finally, the os.File.Close() method is called on the os.File object to close the file and then a loop iterates through and prints each of the slice values. package main import (     "bufio"     "fmt"     "os" ) func main() {     file, _ := os.Open("./file.txt")