Python Notes for Professionals (GoalKicker.com) (Z-Library) (1)

Page 1

Python Notes for ProfessionalsPython® Notes for Professionals GoalKicker.com Free Programming Books Disclaimer This is an unocial free book created for educational purposes and is not aliated with ocial Python® group(s) or company(s). All trademarks and registered trademarks are the property of their respective owners 800+ pages of professional hints and tricks

Page 2

Contents About 1 ................................................................................................................................................................................... Chapter 1: Getting started with Python Language 2 ...................................................................................... Section 1.1: Getting Started 2 ........................................................................................................................................... Section 1.2: Creating variables and assigning values 6 ................................................................................................ Section 1.3: Block Indentation 10 ..................................................................................................................................... Section 1.4: Datatypes 11 ................................................................................................................................................. Section 1.5: Collection Types 15 ...................................................................................................................................... Section 1.6: IDLE - Python GUI 19 .................................................................................................................................... Section 1.7: User Input 21 ................................................................................................................................................. Section 1.8: Built in Modules and Functions 21 .............................................................................................................. Section 1.9: Creating a module 25 ................................................................................................................................... Section 1.10: Installation of Python 2.7.x and 3.x 26 ....................................................................................................... Section 1.11: String function - str() and repr() 28 ........................................................................................................... Section 1.12: Installing external modules using pip 29 ................................................................................................... Section 1.13: Help Utility 31 ............................................................................................................................................... Chapter 2: Python Data Types 33 ............................................................................................................................ Section 2.1: String Data Type 33 ..................................................................................................................................... Section 2.2: Set Data Types 33 ....................................................................................................................................... Section 2.3: Numbers data type 33 ................................................................................................................................ Section 2.4: List Data Type 34 ......................................................................................................................................... Section 2.5: Dictionary Data Type 34 ............................................................................................................................. Section 2.6: Tuple Data Type 34 ..................................................................................................................................... Chapter 3: Indentation 35 ............................................................................................................................................. Section 3.1: Simple example 35 ....................................................................................................................................... Section 3.2: How Indentation is Parsed 35 ..................................................................................................................... Section 3.3: Indentation Errors 36 ................................................................................................................................... Chapter 4: Comments and Documentation 37 .................................................................................................. Section 4.1: Single line, inline and multiline comments 37 ............................................................................................ Section 4.2: Programmatically accessing docstrings 37 .............................................................................................. Section 4.3: Write documentation using docstrings 38 ................................................................................................ Chapter 5: Date and Time 42 ...................................................................................................................................... Section 5.1: Parsing a string into a timezone aware datetime object 42 .................................................................... Section 5.2: Constructing timezone-aware datetimes 42 ............................................................................................ Section 5.3: Computing time dierences 44 .................................................................................................................. Section 5.4: Basic datetime objects usage 44 ............................................................................................................... Section 5.5: Switching between time zones 45 .............................................................................................................. Section 5.6: Simple date arithmetic 45 ........................................................................................................................... Section 5.7: Converting timestamp to datetime 46 ...................................................................................................... Section 5.8: Subtracting months from a date accurately 46 ....................................................................................... Section 5.9: Parsing an arbitrary ISO 8601 timestamp with minimal libraries 46 ...................................................... Section 5.10: Get an ISO 8601 timestamp 47 .................................................................................................................. Section 5.11: Parsing a string with a short time zone name into a timezone aware datetime object 47 ................ Section 5.12: Fuzzy datetime parsing (extracting datetime out of a text) 48 ............................................................ Section 5.13: Iterate over dates 49 .................................................................................................................................. Chapter 6: Date Formatting 50 .................................................................................................................................. Section 6.1: Time between two date-times 50 ............................................................................................................... Section 6.2: Outputting datetime object to string 50 ....................................................................................................

Page 3

Section 6.3: Parsing string to datetime object 50 ......................................................................................................... Chapter 7: Enum 51 .......................................................................................................................................................... Section 7.1: Creating an enum (Python 2.4 through 3.3) 51 ......................................................................................... Section 7.2: Iteration 51 ................................................................................................................................................... Chapter 8: Set 52 ............................................................................................................................................................... Section 8.1: Operations on sets 52 .................................................................................................................................. Section 8.2: Get the unique elements of a list 53 .......................................................................................................... Section 8.3: Set of Sets 53 ................................................................................................................................................ Section 8.4: Set Operations using Methods and Builtins 53 ......................................................................................... Section 8.5: Sets versus multisets 55 .............................................................................................................................. Chapter 9: Simple Mathematical Operators 57 ................................................................................................. Section 9.1: Division 57 ..................................................................................................................................................... Section 9.2: Addition 58 .................................................................................................................................................... Section 9.3: Exponentiation 59 ........................................................................................................................................ Section 9.4: Trigonometric Functions 60 ........................................................................................................................ Section 9.5: Inplace Operations 61 ................................................................................................................................. Section 9.6: Subtraction 61 .............................................................................................................................................. Section 9.7: Multiplication 61 ........................................................................................................................................... Section 9.8: Logarithms 62 .............................................................................................................................................. Section 9.9: Modulus 62 ................................................................................................................................................... Chapter 10: Bitwise Operators 65 ............................................................................................................................. Section 10.1: Bitwise NOT 65 ............................................................................................................................................ Section 10.2: Bitwise XOR (Exclusive OR) 66 .................................................................................................................. Section 10.3: Bitwise AND 67 ............................................................................................................................................ Section 10.4: Bitwise OR 67 .............................................................................................................................................. Section 10.5: Bitwise Left Shift 67 .................................................................................................................................... Section 10.6: Bitwise Right Shift 68 .................................................................................................................................. Section 10.7: Inplace Operations 68 ................................................................................................................................ Chapter 11: Boolean Operators 69 ............................................................................................................................ Section 11.1: `and` and `or` are not guaranteed to return a boolean 69 ...................................................................... Section 11.2: A simple example 69 ................................................................................................................................... Section 11.3: Short-circuit evaluation 69 ......................................................................................................................... Section 11.4: and 70 ........................................................................................................................................................... Section 11.5: or 70 .............................................................................................................................................................. Section 11.6: not 71 ............................................................................................................................................................ Chapter 12: Operator Precedence 72 ...................................................................................................................... Section 12.1: Simple Operator Precedence Examples in python 72 ............................................................................. Chapter 13: Variable Scope and Binding 73 ......................................................................................................... Section 13.1: Nonlocal Variables 73 ................................................................................................................................. Section 13.2: Global Variables 73 .................................................................................................................................... Section 13.3: Local Variables 74 ...................................................................................................................................... Section 13.4: The del command 75 ................................................................................................................................. Section 13.5: Functions skip class scope when looking up names 76 ......................................................................... Section 13.6: Local vs Global Scope 77 ........................................................................................................................... Section 13.7: Binding Occurrence 79 ............................................................................................................................... Chapter 14: Conditionals 80 ......................................................................................................................................... Section 14.1: Conditional Expression (or "The Ternary Operator") 80 ......................................................................... Section 14.2: if, elif, and else 80 ....................................................................................................................................... Section 14.3: Truth Values 80 ...........................................................................................................................................

Page 4

Section 14.4: Boolean Logic Expressions 81 ................................................................................................................... Section 14.5: Using the cmp function to get the comparison result of two objects 83 ............................................. Section 14.6: Else statement 83 ....................................................................................................................................... Section 14.7: Testing if an object is None and assigning it 84 ...................................................................................... Section 14.8: If statement 84 ............................................................................................................................................ Chapter 15: Comparisons 86 ........................................................................................................................................ Section 15.1: Chain Comparisons 86 ................................................................................................................................ Section 15.2: Comparison by `is` vs `==` 87 ...................................................................................................................... Section 15.3: Greater than or less than 88 ...................................................................................................................... Section 15.4: Not equal to 88 ........................................................................................................................................... Section 15.5: Equal To 89 ................................................................................................................................................. Section 15.6: Comparing Objects 89 ............................................................................................................................... Chapter 16: Loops 91 ....................................................................................................................................................... Section 16.1: Break and Continue in Loops 91 ................................................................................................................ Section 16.2: For loops 93 ................................................................................................................................................ Section 16.3: Iterating over lists 93 .................................................................................................................................. Section 16.4: Loops with an "else" clause 94 .................................................................................................................. Section 16.5: The Pass Statement 96 .............................................................................................................................. Section 16.6: Iterating over dictionaries 97 .................................................................................................................... Section 16.7: The "half loop" do-while 98 ........................................................................................................................ Section 16.8: Looping and Unpacking 98 ....................................................................................................................... Section 16.9: Iterating dierent portion of a list with dierent step size 99 ............................................................... Section 16.10: While Loop 100 .......................................................................................................................................... Chapter 17: Arrays 102 .................................................................................................................................................... Section 17.1: Access individual elements through indexes 102 ..................................................................................... Section 17.2: Basic Introduction to Arrays 102 .............................................................................................................. Section 17.3: Append any value to the array using append() method 103 ................................................................ Section 17.4: Insert value in an array using insert() method 103 ................................................................................ Section 17.5: Extend python array using extend() method 103 ................................................................................... Section 17.6: Add items from list into array using fromlist() method 104 .................................................................. Section 17.7: Remove any array element using remove() method 104 ..................................................................... Section 17.8: Remove last array element using pop() method 104 ............................................................................ Section 17.9: Fetch any element through its index using index() method 104 ........................................................... Section 17.10: Reverse a python array using reverse() method 104 ........................................................................... Section 17.11: Get array buer information through buer_info() method 105 ........................................................ Section 17.12: Check for number of occurrences of an element using count() method 105 .................................... Section 17.13: Convert array to string using tostring() method 105 ............................................................................ Section 17.14: Convert array to a python list with same elements using tolist() method 105 .................................. Section 17.15: Append a string to char array using fromstring() method 105 ........................................................... Chapter 18: Multidimensional arrays 106 .............................................................................................................. Section 18.1: Lists in lists 106 ............................................................................................................................................ Section 18.2: Lists in lists in lists in.. 106 .......................................................................................................................... Chapter 19: Dictionary 108 ............................................................................................................................................ Section 19.1: Introduction to Dictionary 108 ................................................................................................................... Section 19.2: Avoiding KeyError Exceptions 109 ........................................................................................................... Section 19.3: Iterating Over a Dictionary 109 ................................................................................................................. Section 19.4: Dictionary with default values 110 ........................................................................................................... Section 19.5: Merging dictionaries 111 ........................................................................................................................... Section 19.6: Accessing keys and values 111 ................................................................................................................ Section 19.7: Accessing values of a dictionary 112 .......................................................................................................

Page 5

Section 19.8: Creating a dictionary 112 .......................................................................................................................... Section 19.9: Creating an ordered dictionary 113 ......................................................................................................... Section 19.10: Unpacking dictionaries using the ** operator 113 ................................................................................. Section 19.11: The trailing comma 114 ............................................................................................................................ Section 19.12: The dict() constructor 114 ........................................................................................................................ Section 19.13: Dictionaries Example 114 ......................................................................................................................... Section 19.14: All combinations of dictionary values 115 .............................................................................................. Chapter 20: List 117 ......................................................................................................................................................... Section 20.1: List methods and supported operators 117 ............................................................................................ Section 20.2: Accessing list values 122 .......................................................................................................................... Section 20.3: Checking if list is empty 123 ..................................................................................................................... Section 20.4: Iterating over a list 123 ............................................................................................................................. Section 20.5: Checking whether an item is in a list 124 ................................................................................................ Section 20.6: Any and All 124 .......................................................................................................................................... Section 20.7: Reversing list elements 125 ...................................................................................................................... Section 20.8: Concatenate and Merge lists 125 ............................................................................................................ Section 20.9: Length of a list 126 .................................................................................................................................... Section 20.10: Remove duplicate values in list 126 ....................................................................................................... Section 20.11: Comparison of lists 127 ............................................................................................................................ Section 20.12: Accessing values in nested list 127 ........................................................................................................ Section 20.13: Initializing a List to a Fixed Number of Elements 128 ........................................................................... Chapter 21: List comprehensions 130 ...................................................................................................................... Section 21.1: List Comprehensions 130 ........................................................................................................................... Section 21.2: Conditional List Comprehensions 132 ...................................................................................................... Section 21.3: Avoid repetitive and expensive operations using conditional clause 134 ............................................ Section 21.4: Dictionary Comprehensions 135 ............................................................................................................... Section 21.5: List Comprehensions with Nested Loops 136 .......................................................................................... Section 21.6: Generator Expressions 138 ........................................................................................................................ Section 21.7: Set Comprehensions 140 ........................................................................................................................... Section 21.8: Refactoring filter and map to list comprehensions 140 ......................................................................... Section 21.9: Comprehensions involving tuples 141 ...................................................................................................... Section 21.10: Counting Occurrences Using Comprehension 142 ............................................................................... Section 21.11: Changing Types in a List 142 .................................................................................................................... Section 21.12: Nested List Comprehensions 142 ............................................................................................................ Section 21.13: Iterate two or more list simultaneously within list comprehension 143 .............................................. Chapter 22: List slicing (selecting parts of lists) 144 ....................................................................................... Section 22.1: Using the third "step" argument 144 ........................................................................................................ Section 22.2: Selecting a sublist from a list 144 ............................................................................................................ Section 22.3: Reversing a list with slicing 144 ................................................................................................................ Section 22.4: Shifting a list using slicing 144 .................................................................................................................. Chapter 23: groupby() 146 ............................................................................................................................................ Section 23.1: Example 4 146 ............................................................................................................................................. Section 23.2: Example 2 146 ............................................................................................................................................ Section 23.3: Example 3 147 ............................................................................................................................................ Chapter 24: Linked lists 149 ......................................................................................................................................... Section 24.1: Single linked list example 149 ................................................................................................................... Chapter 25: Linked List Node 154 ............................................................................................................................. Section 25.1: Write a simple Linked List Node in python 154 ....................................................................................... Chapter 26: Filter 155 ......................................................................................................................................................

Page 6

Section 26.1: Basic use of filter 155 ................................................................................................................................. Section 26.2: Filter without function 155 ........................................................................................................................ Section 26.3: Filter as short-circuit check 156 ............................................................................................................... Section 26.4: Complementary function: filterfalse, ifilterfalse 156 .............................................................................. Chapter 27: Heapq 158 ................................................................................................................................................... Section 27.1: Largest and smallest items in a collection 158 ....................................................................................... Section 27.2: Smallest item in a collection 158 .............................................................................................................. Chapter 28: Tuple 160 ..................................................................................................................................................... Section 28.1: Tuple 160 ..................................................................................................................................................... Section 28.2: Tuples are immutable 161 ........................................................................................................................ Section 28.3: Packing and Unpacking Tuples 161 ........................................................................................................ Section 28.4: Built-in Tuple Functions 162 ..................................................................................................................... Section 28.5: Tuple Are Element-wise Hashable and Equatable 163 ......................................................................... Section 28.6: Indexing Tuples 164 ................................................................................................................................... Section 28.7: Reversing Elements 164 ............................................................................................................................ Chapter 29: Basic Input and Output 165 ............................................................................................................... Section 29.1: Using the print function 165 ...................................................................................................................... Section 29.2: Input from a File 165 ................................................................................................................................. Section 29.3: Read from stdin 167 .................................................................................................................................. Section 29.4: Using input() and raw_input() 167 .......................................................................................................... Section 29.5: Function to prompt user for a number 167 ............................................................................................ Section 29.6: Printing a string without a newline at the end 168 ................................................................................. Chapter 30: Files & Folders I/O 171 ......................................................................................................................... Section 30.1: File modes 171 ............................................................................................................................................ Section 30.2: Reading a file line-by-line 172 .................................................................................................................. Section 30.3: Iterate files (recursively) 173 .................................................................................................................... Section 30.4: Getting the full contents of a file 173 ...................................................................................................... Section 30.5: Writing to a file 174 ................................................................................................................................... Section 30.6: Check whether a file or path exists 175 .................................................................................................. Section 30.7: Random File Access Using mmap 176 .................................................................................................... Section 30.8: Replacing text in a file 176 ....................................................................................................................... Section 30.9: Checking if a file is empty 176 ................................................................................................................. Section 30.10: Read a file between a range of lines 177 .............................................................................................. Section 30.11: Copy a directory tree 177 ........................................................................................................................ Section 30.12: Copying contents of one file to a dierent file 177 .............................................................................. Chapter 31: os.path 178 .................................................................................................................................................. Section 31.1: Join Paths 178 ............................................................................................................................................. Section 31.2: Path Component Manipulation 178 .......................................................................................................... Section 31.3: Get the parent directory 178 ..................................................................................................................... Section 31.4: If the given path exists 178 ........................................................................................................................ Section 31.5: check if the given path is a directory, file, symbolic link, mount point etc 179 .................................... Section 31.6: Absolute Path from Relative Path 179 ..................................................................................................... Chapter 32: Iterables and Iterators 180 ................................................................................................................ Section 32.1: Iterator vs Iterable vs Generator 180 ....................................................................................................... Section 32.2: Extract values one by one 181 ................................................................................................................. Section 32.3: Iterating over entire iterable 181 ............................................................................................................. Section 32.4: Verify only one element in iterable 181 .................................................................................................. Section 32.5: What can be iterable 182 .......................................................................................................................... Section 32.6: Iterator isn't reentrant! 182 .......................................................................................................................

Page 7

Chapter 33: Functions 183 ............................................................................................................................................. Section 33.1: Defining and calling simple functions 183 ............................................................................................... Section 33.2: Defining a function with an arbitrary number of arguments 184 ........................................................ Section 33.3: Lambda (Inline/Anonymous) Functions 187 .......................................................................................... Section 33.4: Defining a function with optional arguments 189 .................................................................................. Section 33.5: Defining a function with optional mutable arguments 190 ................................................................... Section 33.6: Argument passing and mutability 191 .................................................................................................... Section 33.7: Returning values from functions 192 ....................................................................................................... Section 33.8: Closure 192 ................................................................................................................................................. Section 33.9: Forcing the use of named parameters 193 ............................................................................................ Section 33.10: Nested functions 194 ............................................................................................................................... Section 33.11: Recursion limit 194 .................................................................................................................................... Section 33.12: Recursive Lambda using assigned variable 195 ................................................................................... Section 33.13: Recursive functions 195 ........................................................................................................................... Section 33.14: Defining a function with arguments 196 ................................................................................................ Section 33.15: Iterable and dictionary unpacking 196 .................................................................................................. Section 33.16: Defining a function with multiple arguments 198 ................................................................................. Chapter 34: Defining functions with list arguments 199 .............................................................................. Section 34.1: Function and Call 199 ................................................................................................................................. Chapter 35: Functional Programming in Python 201 ...................................................................................... Section 35.1: Lambda Function 201 ................................................................................................................................ Section 35.2: Map Function 201 ...................................................................................................................................... Section 35.3: Reduce Function 201 ................................................................................................................................. Section 35.4: Filter Function 201 ..................................................................................................................................... Chapter 36: Partial functions 202 .............................................................................................................................. Section 36.1: Raise the power 202 ................................................................................................................................... Chapter 37: Decorators 203 ......................................................................................................................................... Section 37.1: Decorator function 203 .............................................................................................................................. Section 37.2: Decorator class 204 ................................................................................................................................... Section 37.3: Decorator with arguments (decorator factory) 205 .............................................................................. Section 37.4: Making a decorator look like the decorated function 207 .................................................................... Section 37.5: Using a decorator to time a function 207 ............................................................................................... Section 37.6: Create singleton class with a decorator 208 .......................................................................................... Chapter 38: Classes 209 ................................................................................................................................................. Section 38.1: Introduction to classes 209 ........................................................................................................................ Section 38.2: Bound, unbound, and static methods 210 .............................................................................................. Section 38.3: Basic inheritance 212 ................................................................................................................................ Section 38.4: Monkey Patching 214 ................................................................................................................................ Section 38.5: New-style vs. old-style classes 214 .......................................................................................................... Section 38.6: Class methods: alternate initializers 215 ................................................................................................. Section 38.7: Multiple Inheritance 217 ............................................................................................................................ Section 38.8: Properties 219 ............................................................................................................................................ Section 38.9: Default values for instance variables 220 ............................................................................................... Section 38.10: Class and instance variables 221 ........................................................................................................... Section 38.11: Class composition 222 .............................................................................................................................. Section 38.12: Listing All Class Members 223 ................................................................................................................. Section 38.13: Singleton class 224 ................................................................................................................................... Section 38.14: Descriptors and Dotted Lookups 225 .................................................................................................... Chapter 39: Metaclasses 226 .......................................................................................................................................

Page 8

Section 39.1: Basic Metaclasses 226 ............................................................................................................................... Section 39.2: Singletons using metaclasses 227 ........................................................................................................... Section 39.3: Using a metaclass 227 .............................................................................................................................. Section 39.4: Introduction to Metaclasses 227 .............................................................................................................. Section 39.5: Custom functionality with metaclasses 228 ........................................................................................... Section 39.6: The default metaclass 229 ....................................................................................................................... Chapter 40: String Formatting 232 ......................................................................................................................... Section 40.1: Basics of String Formatting 232 ............................................................................................................... Section 40.2: Alignment and padding 233 ..................................................................................................................... Section 40.3: Format literals (f-string) 234 .................................................................................................................... Section 40.4: Float formatting 234 ................................................................................................................................. Section 40.5: Named placeholders 235 ......................................................................................................................... Section 40.6: String formatting with datetime 236 ....................................................................................................... Section 40.7: Formatting Numerical Values 236 ........................................................................................................... Section 40.8: Nested formatting 237 .............................................................................................................................. Section 40.9: Format using Getitem and Getattr 237 ................................................................................................... Section 40.10: Padding and truncating strings, combined 237 .................................................................................... Section 40.11: Custom formatting for a class 238 ......................................................................................................... Chapter 41: String Methods 240 ................................................................................................................................ Section 41.1: Changing the capitalization of a string 240 ............................................................................................. Section 41.2: str.translate: Translating characters in a string 241 ............................................................................... Section 41.3: str.format and f-strings: Format values into a string 242 ...................................................................... Section 41.4: String module's useful constants 243 ....................................................................................................... Section 41.5: Stripping unwanted leading/trailing characters from a string 244 ...................................................... Section 41.6: Reversing a string 245 ............................................................................................................................... Section 41.7: Split a string based on a delimiter into a list of strings 245 ................................................................... Section 41.8: Replace all occurrences of one substring with another substring 246 ................................................ Section 41.9: Testing what a string is composed of 247 ............................................................................................... Section 41.10: String Contains 249 ................................................................................................................................... Section 41.11: Join a list of strings into one string 249 ................................................................................................... Section 41.12: Counting number of times a substring appears in a string 250 .......................................................... Section 41.13: Case insensitive string comparisons 250 ................................................................................................ Section 41.14: Justify strings 251 ..................................................................................................................................... Section 41.15: Test the starting and ending characters of a string 252 ...................................................................... Section 41.16: Conversion between str or bytes data and unicode characters 253 .................................................. Chapter 42: Using loops within functions 255 .................................................................................................... Section 42.1: Return statement inside loop in a function 255 ...................................................................................... Chapter 43: Importing modules 256 ........................................................................................................................ Section 43.1: Importing a module 256 ............................................................................................................................ Section 43.2: The __all__ special variable 257 ............................................................................................................ Section 43.3: Import modules from an arbitrary filesystem location 258 .................................................................. Section 43.4: Importing all names from a module 258 ................................................................................................ Section 43.5: Programmatic importing 259 ................................................................................................................... Section 43.6: PEP8 rules for Imports 259 ....................................................................................................................... Section 43.7: Importing specific names from a module 260 ........................................................................................ Section 43.8: Importing submodules 260 ....................................................................................................................... Section 43.9: Re-importing a module 260 ...................................................................................................................... Section 43.10: __import__() function 261 ..................................................................................................................... Chapter 44: Dierence between Module and Package 262 ...................................................................... Section 44.1: Modules 262 ................................................................................................................................................

Page 9

Section 44.2: Packages 262 ............................................................................................................................................. Chapter 45: Math Module 264 .................................................................................................................................... Section 45.1: Rounding: round, floor, ceil, trunc 264 ...................................................................................................... Section 45.2: Trigonometry 265 ...................................................................................................................................... Section 45.3: Pow for faster exponentiation 266 ........................................................................................................... Section 45.4: Infinity and NaN ("not a number") 266 ................................................................................................... Section 45.5: Logarithms 269 .......................................................................................................................................... Section 45.6: Constants 269 ............................................................................................................................................ Section 45.7: Imaginary Numbers 270 ........................................................................................................................... Section 45.8: Copying signs 270 ..................................................................................................................................... Section 45.9: Complex numbers and the cmath module 270 ...................................................................................... Chapter 46: Complex math 273 ................................................................................................................................. Section 46.1: Advanced complex arithmetic 273 ........................................................................................................... Section 46.2: Basic complex arithmetic 274 .................................................................................................................. Chapter 47: Collections module 275 ....................................................................................................................... Section 47.1: collections.Counter 275 .............................................................................................................................. Section 47.2: collections.OrderedDict 276 ...................................................................................................................... Section 47.3: collections.defaultdict 277 ......................................................................................................................... Section 47.4: collections.namedtuple 278 ...................................................................................................................... Section 47.5: collections.deque 279 ................................................................................................................................ Section 47.6: collections.ChainMap 280 .......................................................................................................................... Chapter 48: Operator module 282 ........................................................................................................................... Section 48.1: Itemgetter 282 ............................................................................................................................................ Section 48.2: Operators as alternative to an infix operator 282 ................................................................................. Section 48.3: Methodcaller 282 ....................................................................................................................................... Chapter 49: JSON Module 284 .................................................................................................................................... Section 49.1: Storing data in a file 284 ............................................................................................................................ Section 49.2: Retrieving data from a file 284 ................................................................................................................ Section 49.3: Formatting JSON output 284 ................................................................................................................... Section 49.4: `load` vs `loads`, `dump` vs `dumps` 285 ................................................................................................... Section 49.5: Calling `json.tool` from the command line to pretty-print JSON output 286 ...................................... Section 49.6: JSON encoding custom objects 286 ........................................................................................................ Section 49.7: Creating JSON from Python dict 287 ...................................................................................................... Section 49.8: Creating Python dict from JSON 287 ...................................................................................................... Chapter 50: Sqlite3 Module 289 ................................................................................................................................. Section 50.1: Sqlite3 - Not require separate server process 289 ................................................................................. Section 50.2: Getting the values from the database and Error handling 289 ........................................................... Chapter 51: The os Module 291 ................................................................................................................................... Section 51.1: makedirs - recursive directory creation 291 ............................................................................................ Section 51.2: Create a directory 292 ............................................................................................................................... Section 51.3: Get current directory 292 .......................................................................................................................... Section 51.4: Determine the name of the operating system 292 ................................................................................ Section 51.5: Remove a directory 292 ............................................................................................................................ Section 51.6: Follow a symlink (POSIX) 292 .................................................................................................................... Section 51.7: Change permissions on a file 292 ............................................................................................................. Chapter 52: The locale Module 293 .......................................................................................................................... Section 52.1: Currency Formatting US Dollars Using the locale Module 293 ............................................................. Chapter 53: Itertools Module 294 .............................................................................................................................. Section 53.1: Combinations method in Itertools Module 294 .......................................................................................

Page 10

Section 53.2: itertools.dropwhile 294 .............................................................................................................................. Section 53.3: Zipping two iterators until they are both exhausted 295 ...................................................................... Section 53.4: Take a slice of a generator 295 ............................................................................................................... Section 53.5: Grouping items from an iterable object using a function 296 .............................................................. Section 53.6: itertools.takewhile 297 ............................................................................................................................... Section 53.7: itertools.permutations 297 ........................................................................................................................ Section 53.8: itertools.repeat 298 .................................................................................................................................... Section 53.9: Get an accumulated sum of numbers in an iterable 298 ...................................................................... Section 53.10: Cycle through elements in an iterator 298 ............................................................................................ Section 53.11: itertools.product 298 ................................................................................................................................. Section 53.12: itertools.count 299 .................................................................................................................................... Section 53.13: Chaining multiple iterators together 300 ............................................................................................... Chapter 54: Asyncio Module 301 ............................................................................................................................... Section 54.1: Coroutine and Delegation Syntax 301 ..................................................................................................... Section 54.2: Asynchronous Executors 302 ................................................................................................................... Section 54.3: Using UVLoop 303 ..................................................................................................................................... Section 54.4: Synchronization Primitive: Event 303 ....................................................................................................... Section 54.5: A Simple Websocket 304 .......................................................................................................................... Section 54.6: Common Misconception about asyncio 304 .......................................................................................... Chapter 55: Random module 307 ............................................................................................................................. Section 55.1: Creating a random user password 307 ................................................................................................... Section 55.2: Create cryptographically secure random numbers 307 ....................................................................... Section 55.3: Random and sequences: shue, choice and sample 308 .................................................................... Section 55.4: Creating random integers and floats: randint, randrange, random, and uniform 309 ...................... Section 55.5: Reproducible random numbers: Seed and State 310 ............................................................................ Section 55.6: Random Binary Decision 311 ................................................................................................................... Chapter 56: Functools Module 312 ........................................................................................................................... Section 56.1: partial 312 ................................................................................................................................................... Section 56.2: cmp_to_key 312 ....................................................................................................................................... Section 56.3: lru_cache 312 ............................................................................................................................................. Section 56.4: total_ordering 313 ..................................................................................................................................... Section 56.5: reduce 314 .................................................................................................................................................. Chapter 57: The dis module 315 ................................................................................................................................ Section 57.1: What is Python bytecode? 315 ................................................................................................................. Section 57.2: Constants in the dis module 315 .............................................................................................................. Section 57.3: Disassembling modules 315 ..................................................................................................................... Chapter 58: The base64 Module 317 ....................................................................................................................... Section 58.1: Encoding and Decoding Base64 318 ....................................................................................................... Section 58.2: Encoding and Decoding Base32 319 ....................................................................................................... Section 58.3: Encoding and Decoding Base16 320 ........................................................................................................ Section 58.4: Encoding and Decoding ASCII85 320 ...................................................................................................... Section 58.5: Encoding and Decoding Base85 321 ....................................................................................................... Chapter 59: Queue Module 322 .................................................................................................................................. Section 59.1: Simple example 322 ................................................................................................................................... Chapter 60: Deque Module 324 .................................................................................................................................. Section 60.1: Basic deque using 324 ............................................................................................................................... Section 60.2: Available methods in deque 324 .............................................................................................................. Section 60.3: limit deque size 325 ................................................................................................................................... Section 60.4: Breadth First Search 325 ..........................................................................................................................

Page 11

Chapter 61: Webbrowser Module 326 ...................................................................................................................... Section 61.1: Opening a URL with Default Browser 326 ................................................................................................ Section 61.2: Opening a URL with Dierent Browsers 327 ........................................................................................... Chapter 62: tkinter 328 ................................................................................................................................................... Section 62.1: Geometry Managers 328 ........................................................................................................................... Section 62.2: A minimal tkinter Application 329 ............................................................................................................ Chapter 63: pyautogui module 331 .......................................................................................................................... Section 63.1: Mouse Functions 331 .................................................................................................................................. Section 63.2: Keyboard Functions 331 ........................................................................................................................... Section 63.3: Screenshot And Image Recognition 331 ................................................................................................. Chapter 64: Indexing and Slicing 332 ...................................................................................................................... Section 64.1: Basic Slicing 332 ......................................................................................................................................... Section 64.2: Reversing an object 333 ........................................................................................................................... Section 64.3: Slice assignment 333 ................................................................................................................................. Section 64.4: Making a shallow copy of an array 333 .................................................................................................. Section 64.5: Indexing custom classes: __getitem__, __setitem__ and __delitem__ 334 ................................... Section 64.6: Basic Indexing 335 ..................................................................................................................................... Chapter 65: Plotting with Matplotlib 337 .............................................................................................................. Section 65.1: Plots with Common X-axis but dierent Y-axis : Using twinx() 337 ....................................................... Section 65.2: Plots with common Y-axis and dierent X-axis using twiny() 338 ....................................................... Section 65.3: A Simple Plot in Matplotlib 340 ................................................................................................................. Section 65.4: Adding more features to a simple plot : axis labels, title, axis ticks, grid, and legend 341 ................ Section 65.5: Making multiple plots in the same figure by superimposition similar to MATLAB 342 ...................... Section 65.6: Making multiple Plots in the same figure using plot superimposition with separate plot commands 343 ......................................................................................................................................................... Chapter 66: graph-tool 345 .......................................................................................................................................... Section 66.1: PyDotPlus 345 ............................................................................................................................................. Section 66.2: PyGraphviz 345 .......................................................................................................................................... Chapter 67: Generators 347 ......................................................................................................................................... Section 67.1: Introduction 347 .......................................................................................................................................... Section 67.2: Infinite sequences 349 ............................................................................................................................... Section 67.3: Sending objects to a generator 350 ........................................................................................................ Section 67.4: Yielding all values from another iterable 351 ......................................................................................... Section 67.5: Iteration 351 ............................................................................................................................................... Section 67.6: The next() function 351 ............................................................................................................................. Section 67.7: Coroutines 352 ........................................................................................................................................... Section 67.8: Refactoring list-building code 352 ........................................................................................................... Section 67.9: Yield with recursion: recursively listing all files in a directory 353 ........................................................ Section 67.10: Generator expressions 354 ...................................................................................................................... Section 67.11: Using a generator to find Fibonacci Numbers 354 ............................................................................... Section 67.12: Searching 354 ........................................................................................................................................... Section 67.13: Iterating over generators in parallel 355 ............................................................................................... Chapter 68: Reduce 356 ................................................................................................................................................. Section 68.1: Overview 356 .............................................................................................................................................. Section 68.2: Using reduce 356 ....................................................................................................................................... Section 68.3: Cumulative product 357 ............................................................................................................................ Section 68.4: Non short-circuit variant of any/all 357 ................................................................................................. Chapter 69: Map Function 358 .................................................................................................................................... Section 69.1: Basic use of map, itertools.imap and future_builtins.map 358 .............................................................

Page 12

Section 69.2: Mapping each value in an iterable 358 ................................................................................................... Section 69.3: Mapping values of dierent iterables 359 .............................................................................................. Section 69.4: Transposing with Map: Using "None" as function argument (python 2.x only) 361 .......................... Section 69.5: Series and Parallel Mapping 361 .............................................................................................................. Chapter 70: Exponentiation 365 ................................................................................................................................ Section 70.1: Exponentiation using builtins: ** and pow() 365 ...................................................................................... Section 70.2: Square root: math.sqrt() and cmath.sqrt 365 ......................................................................................... Section 70.3: Modular exponentiation: pow() with 3 arguments 366 .......................................................................... Section 70.4: Computing large integer roots 366 ......................................................................................................... Section 70.5: Exponentiation using the math module: math.pow() 367 ..................................................................... Section 70.6: Exponential function: math.exp() and cmath.exp() 368 ......................................................................... Section 70.7: Exponential function minus 1: math.expm1() 368 .................................................................................... Section 70.8: Magic methods and exponentiation: builtin, math and cmath 369 ...................................................... Section 70.9: Roots: nth-root with fractional exponents 370 ....................................................................................... Chapter 71: Searching 371 ............................................................................................................................................ Section 71.1: Searching for an element 371 .................................................................................................................... Section 71.2: Searching in custom classes: __contains__ and __iter__ 371 ........................................................... Section 71.3: Getting the index for strings: str.index(), str.rindex() and str.find(), str.rfind() 372 ............................... Section 71.4: Getting the index list and tuples: list.index(), tuple.index() 373 .............................................................. Section 71.5: Searching key(s) for a value in dict 373 ................................................................................................... Section 71.6: Getting the index for sorted sequences: bisect.bisect_left() 374 .......................................................... Section 71.7: Searching nested sequences 374 ............................................................................................................. Chapter 72: Sorting, Minimum and Maximum 376 ............................................................................................ Section 72.1: Make custom classes orderable 376 ........................................................................................................ Section 72.2: Special case: dictionaries 378 ................................................................................................................... Section 72.3: Using the key argument 379 .................................................................................................................... Section 72.4: Default Argument to max, min 379 ......................................................................................................... Section 72.5: Getting a sorted sequence 380 ................................................................................................................ Section 72.6: Extracting N largest or N smallest items from an iterable 380 ............................................................ Section 72.7: Getting the minimum or maximum of several values 381 .................................................................... Section 72.8: Minimum and Maximum of a sequence 381 ........................................................................................... Chapter 73: Counting 382 .............................................................................................................................................. Section 73.1: Counting all occurrence of all items in an iterable: collections.Counter 382 ........................................ Section 73.2: Getting the most common value(-s): collections.Counter.most_common() 382 ................................ Section 73.3: Counting the occurrences of one item in a sequence: list.count() and tuple.count() 382 .................. Section 73.4: Counting the occurrences of a substring in a string: str.count() 383 ................................................... Section 73.5: Counting occurrences in numpy array 383 ............................................................................................ Chapter 74: The Print Function 384 ......................................................................................................................... Section 74.1: Print basics 384 ........................................................................................................................................... Section 74.2: Print parameters 385 ................................................................................................................................ Chapter 75: Regular Expressions (Regex) 388 ................................................................................................... Section 75.1: Matching the beginning of a string 388 ................................................................................................... Section 75.2: Searching 389 ............................................................................................................................................ Section 75.3: Precompiled patterns 389 ......................................................................................................................... Section 75.4: Flags 390 .................................................................................................................................................... Section 75.5: Replacing 391 ............................................................................................................................................. Section 75.6: Find All Non-Overlapping Matches 391 ................................................................................................... Section 75.7: Checking for allowed characters 392 ...................................................................................................... Section 75.8: Splitting a string using regular expressions 392 ..................................................................................... Section 75.9: Grouping 392 ..............................................................................................................................................

Page 13

Section 75.10: Escaping Special Characters 393 ........................................................................................................... Section 75.11: Match an expression only in specific locations 394 ............................................................................... Section 75.12: Iterating over matches using `re.finditer` 395 ........................................................................................ Chapter 76: Copying data 396 .................................................................................................................................... Section 76.1: Copy a dictionary 396 ................................................................................................................................ Section 76.2: Performing a shallow copy 396 ............................................................................................................... Section 76.3: Performing a deep copy 396 .................................................................................................................... Section 76.4: Performing a shallow copy of a list 396 .................................................................................................. Section 76.5: Copy a set 396 ........................................................................................................................................... Chapter 77: Context Managers (“with” Statement) 398 ............................................................................... Section 77.1: Introduction to context managers and the with statement 398 ............................................................ Section 77.2: Writing your own context manager 398 ................................................................................................. Section 77.3: Writing your own contextmanager using generator syntax 399 ......................................................... Section 77.4: Multiple context managers 400 ................................................................................................................ Section 77.5: Assigning to a target 400 .......................................................................................................................... Section 77.6: Manage Resources 401 ............................................................................................................................. Chapter 78: The __name__ special variable 402 ........................................................................................... Section 78.1: __name__ == '__main__' 402 ................................................................................................................. Section 78.2: Use in logging 402 ..................................................................................................................................... Section 78.3: function_class_or_module.__name__ 402 .......................................................................................... Chapter 79: Checking Path Existence and Permissions 404 ......................................................................... Section 79.1: Perform checks using os.access 404 ........................................................................................................ Chapter 80: Creating Python packages 406 ....................................................................................................... Section 80.1: Introduction 406 ......................................................................................................................................... Section 80.2: Uploading to PyPI 406 .............................................................................................................................. Section 80.3: Making package executable 408 ............................................................................................................. Chapter 81: Usage of "pip" module: PyPI Package Manager 410 ............................................................. Section 81.1: Example use of commands 410 ................................................................................................................ Section 81.2: Handling ImportError Exception 410 ........................................................................................................ Section 81.3: Force install 411 .......................................................................................................................................... Chapter 82: pip: PyPI Package Manager 412 ...................................................................................................... Section 82.1: Install Packages 412 .................................................................................................................................. Section 82.2: To list all packages installed using `pip` 412 ........................................................................................... Section 82.3: Upgrade Packages 412 ............................................................................................................................ Section 82.4: Uninstall Packages 413 ............................................................................................................................. Section 82.5: Updating all outdated packages on Linux 413 ...................................................................................... Section 82.6: Updating all outdated packages on Windows 413 ................................................................................ Section 82.7: Create a requirements.txt file of all packages on the system 413 ....................................................... Section 82.8: Using a certain Python version with pip 414 .......................................................................................... Section 82.9: Create a requirements.txt file of packages only in the current virtualenv 414 .................................. Section 82.10: Installing packages not yet on pip as wheels 415 ................................................................................ Chapter 83: Parsing Command Line arguments 418 ...................................................................................... Section 83.1: Hello world in argparse 418 ...................................................................................................................... Section 83.2: Using command line arguments with argv 418 ..................................................................................... Section 83.3: Setting mutually exclusive arguments with argparse 419 .................................................................... Section 83.4: Basic example with docopt 420 ............................................................................................................... Section 83.5: Custom parser error message with argparse 420 ................................................................................. Section 83.6: Conceptual grouping of arguments with argparse.add_argument_group() 421 ............................. Section 83.7: Advanced example with docopt and docopt_dispatch 422 .................................................................

Page 14

Chapter 84: Subprocess Library 424 ...................................................................................................................... Section 84.1: More flexibility with Popen 424 ................................................................................................................. Section 84.2: Calling External Commands 425 .............................................................................................................. Section 84.3: How to create the command list argument 425 .................................................................................... Chapter 85: setup.py 427 .............................................................................................................................................. Section 85.1: Purpose of setup.py 427 ............................................................................................................................ Section 85.2: Using source control metadata in setup.py 427 .................................................................................... Section 85.3: Adding command line scripts to your python package 428 ................................................................. Section 85.4: Adding installation options 428 ................................................................................................................ Chapter 86: Recursion 430 ............................................................................................................................................ Section 86.1: The What, How, and When of Recursion 430 .......................................................................................... Section 86.2: Tree exploration with recursion 433 ........................................................................................................ Section 86.3: Sum of numbers from 1 to n 434 .............................................................................................................. Section 86.4: Increasing the Maximum Recursion Depth 434 ...................................................................................... Section 86.5: Tail Recursion - Bad Practice 435 ............................................................................................................ Section 86.6: Tail Recursion Optimization Through Stack Introspection 435 ............................................................ Chapter 87: Type Hints 437 .......................................................................................................................................... Section 87.1: Adding types to a function 437 ................................................................................................................. Section 87.2: NamedTuple 438 ....................................................................................................................................... Section 87.3: Generic Types 438 ..................................................................................................................................... Section 87.4: Variables and Attributes 438 .................................................................................................................... Section 87.5: Class Members and Methods 439 ............................................................................................................ Section 87.6: Type hints for keyword arguments 439 .................................................................................................. Chapter 88: Exceptions 440 .......................................................................................................................................... Section 88.1: Catching Exceptions 440 ............................................................................................................................ Section 88.2: Do not catch everything! 440 ................................................................................................................... Section 88.3: Re-raising exceptions 441 ......................................................................................................................... Section 88.4: Catching multiple exceptions 441 ............................................................................................................ Section 88.5: Exception Hierarchy 442 ........................................................................................................................... Section 88.6: Else 444 ....................................................................................................................................................... Section 88.7: Raising Exceptions 444 .............................................................................................................................. Section 88.8: Creating custom exception types 445 ..................................................................................................... Section 88.9: Practical examples of exception handling 445 ....................................................................................... Section 88.10: Exceptions are Objects too 446 .............................................................................................................. Section 88.11: Running clean-up code with finally 446 .................................................................................................. Section 88.12: Chain exceptions with raise from 447 .................................................................................................... Chapter 89: Raise Custom Errors / Exceptions 448 ......................................................................................... Section 89.1: Custom Exception 448 ............................................................................................................................... Section 89.2: Catch custom Exception 448 .................................................................................................................... Chapter 90: Commonwealth Exceptions 450 ....................................................................................................... Section 90.1: Other Errors 450 ......................................................................................................................................... Section 90.2: NameError: name ' ' is not defined 451 .............................................................................................. Section 90.3: TypeErrors 452 .......................................................................................................................................... Section 90.4: Syntax Error on good code 453 ............................................................................................................... Section 90.5: IndentationErrors (or indentation SyntaxErrors) 454 ............................................................................ Chapter 91: urllib 456 ....................................................................................................................................................... Section 91.1: HTTP GET 456 .............................................................................................................................................. Section 91.2: HTTP POST 456 .......................................................................................................................................... Section 91.3: Decode received bytes according to content type encoding 457 ........................................................

Page 15

Chapter 92: Web scraping with Python 458 ......................................................................................................... Section 92.1: Scraping using the Scrapy framework 458 ............................................................................................. Section 92.2: Scraping using Selenium WebDriver 458 ................................................................................................ Section 92.3: Basic example of using requests and lxml to scrape some data 459 ................................................. Section 92.4: Maintaining web-scraping session with requests 459 ........................................................................... Section 92.5: Scraping using BeautifulSoup4 460 ......................................................................................................... Section 92.6: Simple web content download with urllib.request 460 .......................................................................... Section 92.7: Modify Scrapy user agent 460 ................................................................................................................. Section 92.8: Scraping with curl 460 ............................................................................................................................... Chapter 93: HTML Parsing 462 .................................................................................................................................... Section 93.1: Using CSS selectors in BeautifulSoup 462 ................................................................................................ Section 93.2: PyQuery 462 ............................................................................................................................................... Section 93.3: Locate a text after an element in BeautifulSoup 463 ............................................................................ Chapter 94: Manipulating XML 464 .......................................................................................................................... Section 94.1: Opening and reading using an ElementTree 464 ................................................................................... Section 94.2: Create and Build XML Documents 464 .................................................................................................... Section 94.3: Modifying an XML File 465 ........................................................................................................................ Section 94.4: Searching the XML with XPath 465 .......................................................................................................... Section 94.5: Opening and reading large XML files using iterparse (incremental parsing) 466 ............................. Chapter 95: Python Requests Post 468 .................................................................................................................. Section 95.1: Simple Post 468 .......................................................................................................................................... Section 95.2: Form Encoded Data 469 ........................................................................................................................... Section 95.3: File Upload 469 .......................................................................................................................................... Section 95.4: Responses 470 ........................................................................................................................................... Section 95.5: Authentication 470 ..................................................................................................................................... Section 95.6: Proxies 471 ................................................................................................................................................. Chapter 96: Distribution 473 ........................................................................................................................................ Section 96.1: py2app 473 ................................................................................................................................................. Section 96.2: cx_Freeze 474 ............................................................................................................................................ Chapter 97: Property Objects 475 ............................................................................................................................ Section 97.1: Using the @property decorator for read-write properties 475 ............................................................. Section 97.2: Using the @property decorator 475 ....................................................................................................... Section 97.3: Overriding just a getter, setter or a deleter of a property object 476 ................................................. Section 97.4: Using properties without decorators 476 ................................................................................................ Chapter 98: Overloading 479 ...................................................................................................................................... Section 98.1: Operator overloading 479 ......................................................................................................................... Section 98.2: Magic/Dunder Methods 480 .................................................................................................................... Section 98.3: Container and sequence types 481 ......................................................................................................... Section 98.4: Callable types 482 ..................................................................................................................................... Section 98.5: Handling unimplemented behaviour 482 ................................................................................................ Chapter 99: Polymorphism 484 .................................................................................................................................. Section 99.1: Duck Typing 484 ......................................................................................................................................... Section 99.2: Basic Polymorphism 484 .......................................................................................................................... Chapter 100: Method Overriding 488 ...................................................................................................................... Section 100.1: Basic method overriding 488 ................................................................................................................... Chapter 101: User-Defined Methods 489 ................................................................................................................ Section 101.1: Creating user-defined method objects 489 ............................................................................................ Section 101.2: Turtle example 490 ................................................................................................................................... Chapter 102: String representations of class instances: __str__ and __repr__

Page 16

methods 491 ........................................................................................................................................................................ Section 102.1: Motivation 491 ........................................................................................................................................... Section 102.2: Both methods implemented, eval-round-trip style __repr__() 495 .................................................. Chapter 103: Debugging 496 ........................................................................................................................................ Section 103.1: Via IPython and ipdb 496 ......................................................................................................................... Section 103.2: The Python Debugger: Step-through Debugging with _pdb_ 496 .................................................... Section 103.3: Remote debugger 498 ............................................................................................................................. Chapter 104: Reading and Writing CSV 499 ........................................................................................................ Section 104.1: Using pandas 499 ..................................................................................................................................... Section 104.2: Writing a TSV file 499 .............................................................................................................................. Chapter 105: Writing to CSV from String or List 501 ...................................................................................... Section 105.1: Basic Write Example 501 .......................................................................................................................... Section 105.2: Appending a String as a newline in a CSV file 501 ............................................................................... Chapter 106: Dynamic code execution with `exec` and `eval` 502 ............................................................. Section 106.1: Executing code provided by untrusted user using exec, eval, or ast.literal_eval 502 ....................... Section 106.2: Evaluating a string containing a Python literal with ast.literal_eval 502 ........................................... Section 106.3: Evaluating statements with exec 502 ..................................................................................................... Section 106.4: Evaluating an expression with eval 503 ................................................................................................. Section 106.5: Precompiling an expression to evaluate it multiple times 503 ............................................................ Section 106.6: Evaluating an expression with eval using custom globals 503 ........................................................... Chapter 107: PyInstaller - Distributing Python Code 504 .............................................................................. Section 107.1: Installation and Setup 504 ........................................................................................................................ Section 107.2: Using Pyinstaller 504 ................................................................................................................................ Section 107.3: Bundling to One Folder 505 ..................................................................................................................... Section 107.4: Bundling to a Single File 505 ................................................................................................................... Chapter 108: Data Visualization with Python 506 ............................................................................................. Section 108.1: Seaborn 506 .............................................................................................................................................. Section 108.2: Matplotlib 508 ........................................................................................................................................... Section 108.3: Plotly 509 ................................................................................................................................................... Section 108.4: MayaVI 511 ............................................................................................................................................... Chapter 109: The Interpreter (Command Line Console) 513 ....................................................................... Section 109.1: Getting general help 513 .......................................................................................................................... Section 109.2: Referring to the last expression 513 ...................................................................................................... Section 109.3: Opening the Python console 514 ............................................................................................................ Section 109.4: The PYTHONSTARTUP variable 514 ...................................................................................................... Section 109.5: Command line arguments 514 ............................................................................................................... Section 109.6: Getting help about an object 515 ........................................................................................................... Chapter 110: *args and **kwargs 518 ....................................................................................................................... Section 110.1: Using **kwargs when writing functions 518 ............................................................................................ Section 110.2: Using *args when writing functions 518 .................................................................................................. Section 110.3: Populating kwarg values with a dictionary 519 ..................................................................................... Section 110.4: Keyword-only and Keyword-required arguments 519 ........................................................................ Section 110.5: Using **kwargs when calling functions 519 ............................................................................................ Section 110.6: **kwargs and default values 519 ............................................................................................................. Section 110.7: Using *args when calling functions 520 .................................................................................................. Chapter 111: Garbage Collection 521 ........................................................................................................................ Section 111.1: Reuse of primitive objects 521 .................................................................................................................. Section 111.2: Eects of the del command 521 .............................................................................................................. Section 111.3: Reference Counting 522 ............................................................................................................................

Page 17

Section 111.4: Garbage Collector for Reference Cycles 522 ......................................................................................... Section 111.5: Forcefully deallocating objects 523 ......................................................................................................... Section 111.6: Viewing the refcount of an object 524 ..................................................................................................... Section 111.7: Do not wait for the garbage collection to clean up 524 ........................................................................ Section 111.8: Managing garbage collection 524 ........................................................................................................... Chapter 112: Pickle data serialisation 526 ............................................................................................................. Section 112.1: Using Pickle to serialize and deserialize an object 526 .......................................................................... Section 112.2: Customize Pickled Data 526 .................................................................................................................... Chapter 113: Binary Data 528 ...................................................................................................................................... Section 113.1: Format a list of values into a byte object 528 ........................................................................................ Section 113.2: Unpack a byte object according to a format string 528 ...................................................................... Section 113.3: Packing a structure 528 ............................................................................................................................ Chapter 114: Idioms 530 .................................................................................................................................................. Section 114.1: Dictionary key initializations 530 .............................................................................................................. Section 114.2: Switching variables 530 ............................................................................................................................ Section 114.3: Use truth value testing 530 ...................................................................................................................... Section 114.4: Test for "__main__" to avoid unexpected code execution 531 .......................................................... Chapter 115: Data Serialization 533 .......................................................................................................................... Section 115.1: Serialization using JSON 533 .................................................................................................................... Section 115.2: Serialization using Pickle 533 ................................................................................................................... Chapter 116: Multiprocessing 535 ............................................................................................................................... Section 116.1: Running Two Simple Processes 535 ......................................................................................................... Section 116.2: Using Pool and Map 535 ........................................................................................................................... Chapter 117: Multithreading 537 ................................................................................................................................. Section 117.1: Basics of multithreading 537 .................................................................................................................... Section 117.2: Communicating between threads 538 .................................................................................................... Section 117.3: Creating a worker pool 539 ...................................................................................................................... Section 117.4: Advanced use of multithreads 539 .......................................................................................................... Section 117.5: Stoppable Thread with a while Loop 541 ............................................................................................... Chapter 118: Processes and Threads 542 .............................................................................................................. Section 118.1: Global Interpreter Lock 542 ...................................................................................................................... Section 118.2: Running in Multiple Threads 543 ............................................................................................................. Section 118.3: Running in Multiple Processes 544 .......................................................................................................... Section 118.4: Sharing State Between Threads 544 ....................................................................................................... Section 118.5: Sharing State Between Processes 545 .................................................................................................... Chapter 119: Python concurrency 546 ..................................................................................................................... Section 119.1: The multiprocessing module 546 ............................................................................................................. Section 119.2: The threading module 547 ....................................................................................................................... Section 119.3: Passing data between multiprocessing processes 547 ........................................................................ Chapter 120: Parallel computation 550 .................................................................................................................. Section 120.1: Using the multiprocessing module to parallelise tasks 550 ................................................................. Section 120.2: Using a C-extension to parallelize tasks 550 ........................................................................................ Section 120.3: Using Parent and Children scripts to execute code in parallel 550 .................................................... Section 120.4: Using PyPar module to parallelize 551 .................................................................................................. Chapter 121: Sockets 552 ................................................................................................................................................ Section 121.1: Raw Sockets on Linux 552 ......................................................................................................................... Section 121.2: Sending data via UDP 552 ....................................................................................................................... Section 121.3: Receiving data via UDP 553 ..................................................................................................................... Section 121.4: Sending data via TCP 553 ........................................................................................................................

Page 18

Section 121.5: Multi-threaded TCP Socket Server 553 ................................................................................................... Chapter 122: Websockets 556 ...................................................................................................................................... Section 122.1: Simple Echo with aiohttp 556 ................................................................................................................... Section 122.2: Wrapper Class with aiohttp 556 ............................................................................................................. Section 122.3: Using Autobahn as a Websocket Factory 557 ...................................................................................... Chapter 123: Sockets And Message Encryption/Decryption Between Client and Server 559 ............................................................................................................................................................................................ Section 123.1: Server side Implementation 559 .............................................................................................................. Section 123.2: Client side Implementation 561 .............................................................................................................. Chapter 124: Python Networking 563 ..................................................................................................................... Section 124.1: Creating a Simple Http Server 563 .......................................................................................................... Section 124.2: Creating a TCP server 563 ...................................................................................................................... Section 124.3: Creating a UDP Server 564 ..................................................................................................................... Section 124.4: Start Simple HttpServer in a thread and open the browser 564 ......................................................... Section 124.5: The simplest Python socket client-server example 565 ....................................................................... Chapter 125: Python HTTP Server 567 .................................................................................................................... Section 125.1: Running a simple HTTP server 567 ......................................................................................................... Section 125.2: Serving files 567 ........................................................................................................................................ Section 125.3: Basic handling of GET, POST, PUT using BaseHTTPRequestHandler 568 ......................................... Section 125.4: Programmatic API of SimpleHTTPServer 569 ....................................................................................... Chapter 126: Flask 571 .................................................................................................................................................... Section 126.1: Files and Templates 571 ........................................................................................................................... Section 126.2: The basics 571 .......................................................................................................................................... Section 126.3: Routing URLs 572 ..................................................................................................................................... Section 126.4: HTTP Methods 573 ................................................................................................................................... Section 126.5: Jinja Templating 573 ............................................................................................................................... Section 126.6: The Request Object 574 .......................................................................................................................... Chapter 127: Introduction to RabbitMQ using AMQPStorm 576 ................................................................ Section 127.1: How to consume messages from RabbitMQ 576 .................................................................................. Section 127.2: How to publish messages to RabbitMQ 577 ......................................................................................... Section 127.3: How to create a delayed queue in RabbitMQ 577 ................................................................................ Chapter 128: Descriptor 580 ......................................................................................................................................... Section 128.1: Simple descriptor 580 ............................................................................................................................... Section 128.2: Two-way conversions 581 ....................................................................................................................... Chapter 129: tempfile NamedTemporaryFile 582 ............................................................................................. Section 129.1: Create (and write to a) known, persistent temporary file 582 ............................................................. Chapter 130: Input, Subset and Output External Data Files using Pandas 584 ................................. Section 130.1: Basic Code to Import, Subset and Write External Data Files Using Pandas 584 ............................... Chapter 131: Unzipping Files 586 ................................................................................................................................ Section 131.1: Using Python ZipFile.extractall() to decompress a ZIP file 586 ............................................................ Section 131.2: Using Python TarFile.extractall() to decompress a tarball 586 ........................................................... Chapter 132: Working with ZIP archives 587 ........................................................................................................ Section 132.1: Examining Zipfile Contents 587 ............................................................................................................... Section 132.2: Opening Zip Files 587 ............................................................................................................................... Section 132.3: Extracting zip file contents to a directory 588 ....................................................................................... Section 132.4: Creating new archives 588 ...................................................................................................................... Chapter 133: Getting start with GZip 589 .............................................................................................................. Section 133.1: Read and write GNU zip files 589 ............................................................................................................

Page 19

Chapter 134: Stack 590 ................................................................................................................................................... Section 134.1: Creating a Stack class with a List Object 590 ........................................................................................ Section 134.2: Parsing Parentheses 591 ......................................................................................................................... Chapter 135: Working around the Global Interpreter Lock (GIL) 593 ..................................................... Section 135.1: Multiprocessing.Pool 593 .......................................................................................................................... Section 135.2: Cython nogil: 594 ...................................................................................................................................... Chapter 136: Deployment 595 ..................................................................................................................................... Section 136.1: Uploading a Conda Package 595 ............................................................................................................ Chapter 137: Logging 597 .............................................................................................................................................. Section 137.1: Introduction to Python Logging 597 ........................................................................................................ Section 137.2: Logging exceptions 598 ........................................................................................................................... Chapter 138: Web Server Gateway Interface (WSGI) 601 ............................................................................. Section 138.1: Server Object (Method) 601 ..................................................................................................................... Chapter 139: Python Server Sent Events 602 ...................................................................................................... Section 139.1: Flask SSE 602 ............................................................................................................................................. Section 139.2: Asyncio SSE 602 ........................................................................................................................................ Chapter 140: Alternatives to switch statement from other languages 604 ....................................... Section 140.1: Use what the language oers: the if/else construct 604 ..................................................................... Section 140.2: Use a dict of functions 604 ...................................................................................................................... Section 140.3: Use class introspection 605 ..................................................................................................................... Section 140.4: Using a context manager 606 ................................................................................................................ Chapter 141: List destructuring (aka packing and unpacking) 607 ......................................................... Section 141.1: Destructuring assignment 607 ................................................................................................................. Section 141.2: Packing function arguments 608 ............................................................................................................ Section 141.3: Unpacking function arguments 610 ........................................................................................................ Chapter 142: Accessing Python source code and bytecode 611 .............................................................. Section 142.1: Display the bytecode of a function 611 .................................................................................................. Section 142.2: Display the source code of an object 611 ............................................................................................. Section 142.3: Exploring the code object of a function 612 .......................................................................................... Chapter 143: Mixins 613 .................................................................................................................................................. Section 143.1: Mixin 613 ..................................................................................................................................................... Section 143.2: Overriding Methods in Mixins 614 ........................................................................................................... Chapter 144: Attribute Access 615 ........................................................................................................................... Section 144.1: Basic Attribute Access using the Dot Notation 615 ............................................................................... Section 144.2: Setters, Getters & Properties 615 ............................................................................................................ Chapter 145: ArcPy 618 .................................................................................................................................................. Section 145.1: createDissolvedGDB to create a file gdb on the workspace 618 ........................................................ Section 145.2: Printing one field's value for all rows of feature class in file geodatabase using Search Cursor 618 ................................................................................................................................................................. Chapter 146: Abstract Base Classes (abc) 619 ................................................................................................... Section 146.1: Setting the ABCMeta metaclass 619 ....................................................................................................... Section 146.2: Why/How to use ABCMeta and @abstractmethod 619 ...................................................................... Chapter 147: Plugin and Extension Classes 621 ................................................................................................. Section 147.1: Mixins 621 ................................................................................................................................................... Section 147.2: Plugins with Customized Classes 622 ..................................................................................................... Chapter 148: Immutable datatypes(int, float, str, tuple and frozensets) 624 .................................. Section 148.1: Individual characters of strings are not assignable 624 ....................................................................... Section 148.2: Tuple's individual members aren't assignable 624 ...............................................................................

Page 20

Section 148.3: Frozenset's are immutable and not assignable 624 ............................................................................. Chapter 149: Incompatibilities moving from Python 2 to Python 3 625 ................................................ Section 149.1: Integer Division 625 ................................................................................................................................... Section 149.2: Unpacking Iterables 626 .......................................................................................................................... Section 149.3: Strings: Bytes versus Unicode 628 .......................................................................................................... Section 149.4: Print statement vs. Print function 630 .................................................................................................... Section 149.5: Dierences between range and xrange functions 631 ........................................................................ Section 149.6: Raising and handling Exceptions 632 ..................................................................................................... Section 149.7: Leaked variables in list comprehension 634 .......................................................................................... Section 149.8: True, False and None 635 ........................................................................................................................ Section 149.9: User Input 635 ........................................................................................................................................... Section 149.10: Comparison of dierent types 636 ....................................................................................................... Section 149.11: .next() method on iterators renamed 636 ............................................................................................. Section 149.12: filter(), map() and zip() return iterators instead of sequences 637 ................................................... Section 149.13: Renamed modules 637 ........................................................................................................................... Section 149.14: Removed operators <> and ``, synonymous with != and repr() 638 ................................................... Section 149.15: long vs. int 638 ......................................................................................................................................... Section 149.16: All classes are "new-style classes" in Python 3 639 ............................................................................ Section 149.17: Reduce is no longer a built-in 640 ......................................................................................................... Section 149.18: Absolute/Relative Imports 640 .............................................................................................................. Section 149.19: map() 642 ................................................................................................................................................ Section 149.20: The round() function tie-breaking and return type 643 .................................................................... Section 149.21: File I/O 644 .............................................................................................................................................. Section 149.22: cmp function removed in Python 3 644 ............................................................................................... Section 149.23: Octal Constants 645 ............................................................................................................................... Section 149.24: Return value when writing to a file object 645 .................................................................................... Section 149.25: exec statement is a function in Python 3 645 ...................................................................................... Section 149.26: encode/decode to hex no longer available 646 ................................................................................. Section 149.27: Dictionary method changes 647 .......................................................................................................... Section 149.28: Class Boolean Value 647 ....................................................................................................................... Section 149.29: hasattr function bug in Python 2 648 ................................................................................................... Chapter 150: 2to3 tool 650 ............................................................................................................................................ Section 150.1: Basic Usage 650 ........................................................................................................................................ Chapter 151: Non-ocial Python implementations 652 ................................................................................ Section 151.1: IronPython 652 ........................................................................................................................................... Section 151.2: Jython 652 ................................................................................................................................................. Section 151.3: Transcrypt 653 .......................................................................................................................................... Chapter 152: Abstract syntax tree 656 ................................................................................................................... Section 152.1: Analyze functions in a python script 656 ................................................................................................ Chapter 153: Unicode and bytes 658 ....................................................................................................................... Section 153.1: Encoding/decoding error handling 658 .................................................................................................. Section 153.2: File I/O 658 ................................................................................................................................................ Section 153.3: Basics 659 .................................................................................................................................................. Chapter 154: Python Serial Communication (pyserial) 661 ......................................................................... Section 154.1: Initialize serial device 661 ......................................................................................................................... Section 154.2: Read from serial port 661 ....................................................................................................................... Section 154.3: Check what serial ports are available on your machine 661 .............................................................. Chapter 155: Neo4j and Cypher using Py2Neo 664 ......................................................................................... Section 155.1: Adding Nodes to Neo4j Graph 664 ..........................................................................................................