Real-World Python A Hackers Guide to Solving Problems with Code (Lee Vaughan) (Z-Library)

L E E V A U G H A N R E A L W O R L D P Y T H O N A H A C K E R ’ S G U I D E T O S O L V I N G P R O B L E M S W I T H C O D E

REAL-WORLD PYTHON

R E A L - W O R L D P Y T H O N A H a c k e r ’ s G u i d e t o S o l v i n g P r o b l e m s w i t h C o d e by Lee Vaughan San Francisco

REAL-WORLD PYTHON. Copyright © 2021 by Lee Vaughan. All rights reserved. No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval system, without the prior written permission of the copyright owner and the publisher. ISBN-13: 978-1-7185-0062-4 (print) ISBN-13: 978-1-7185-0063-1 (ebook) Publisher: William Pollock Executive Editor: Barbara Yien Production Editor: Kassie Andreadis Developmental Editor: Frances Saux Project Editor: Dapinder Dosanjh Cover Illustrator: Rob Gale Interior Design: Octopod Studios Technical Reviewers: Chris Kren and Eric Mortenson Copyeditor: Kim Wimpsett Compositor: Shawn Morningstar Proofreader: Paula L. Fleming Indexer: Beth Nauman-Montana The following images are reproduced with permission: Figure 3-3 from istockphoto.com; Figure 5-1 courtesy of Lowell Observatory Archives; Figures 5-2, 6-2, 7-6, 7-7, 8-18, and 11-2 courtesy of Wikimedia Commons; Figures 7-2, 7-9, 7-17, 8-20, and 11-1 courtesy of NASA; Figure 8-1 photo by Evan Clark; Figure 8-4 photo by author; Figure 9-5 from pixelsquid.com; Figure 11-9 photo by Hannah Vaughan For information on distribution, translations, or bulk sales, please contact No Starch Press, Inc. directly: No Starch Press, Inc. 245 8th Street, San Francisco, CA 94103 phone: 1-415-863-9900; info@nostarch.com www.nostarch.com Library of Congress Cataloging-in-Publication Data Names: Vaughan, Lee, author. Title: Real-world python: a hacker’s guide to solving problems with code / Lee Vaughan. Description: San Francisco, CA : No Starch Press, Inc., [2020] | Includes index. Identifiers: LCCN 2020022671 (print) | LCCN 2020022672 (ebook) | ISBN 9781718500624 (paperback) | ISBN 1718500629 (paperback) | ISBN 9781718500631 (ebook) Subjects: LCSH: Python (Computer program language) Classification: LCC QA76.73.P98 V383 2020 (print) | LCC QA76.73.P98 (ebook) | DDC 005.1/33--dc23 LC record available at https://lccn.loc.gov/2020022671 LC ebook record available at https://lccn.loc.gov/2020022672 No Starch Press and the No Starch Press logo are registered trademarks of No Starch Press, Inc. Other prod- uct and company names mentioned herein may be the trademarks of their respective owners. Rather than use a trademark symbol with every occurrence of a trademarked name, we are using the names only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark. The information in this book is distributed on an “As Is” basis, without warranty. While every precaution has been taken in the preparation of this work, neither the author nor No Starch Press, Inc. shall have any liability to any person or entity with respect to any loss or damage caused or alleged to be caused directly or indirectly by the information contained in it.

For my uncle, Kenneth P. Vaughan. He brightened every room he entered.

About the Author Lee Vaughan is a programmer, pop culture enthusiast, educator, and author of Impractical Python Projects (No Starch Press, 2018). As an executive-level scientist at ExxonMobil, he constructed and reviewed computer models, developed and tested software, and trained geoscientists and engineers. He wrote both Impractical Python Projects and Real-World Python to help self-learners hone their Python skills and have fun doing it! About the Technical Reviewers Chris Kren graduated from the University of South Alabama with an M.S. in Information Systems. He currently works in the field of cybersecurity and often uses Python for reporting, data analysis, and automation. Eric Mortenson has a PhD in mathematics from the University of Wisconsin at Madison. He has held research and teaching positions at The Pennsylvania State University, The University of Queensland, and the Max Planck Institute for Mathematics. He is an associate professor in mathematics at St. Petersburg State University.

B R I E F C O N T E N T S Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix Chapter 1: Saving Shipwrecked Sailors with Bayes’ Rule . . . . . . . . . . . . . . . . . . . . . . . 1 Chapter 2: Attributing Authorship with Stylometry . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 Chapter 3: Summarizing Speeches with Natural Language Processing . . . . . . . . . . . . . 51 Chapter 4: Sending Super-Secret Messages with a Book Cipher . . . . . . . . . . . . . . . . . 77 Chapter 5: Finding Pluto . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 Chapter 6: Winning the Moon Race with Apollo 8 . . . . . . . . . . . . . . . . . . . . . . . . . . 123 Chapter 7: Selecting Martian Landing Sites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 Chapter 8: Detecting Distant Exoplanets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177 Chapter 9: Identifying Friend or Foe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203 Chapter 10: Restricting Access with Face Recognition . . . . . . . . . . . . . . . . . . . . . . . 225 Chapter 11: Creating an Interactive Zombie Escape Map . . . . . . . . . . . . . . . . . . . . . 245 Chapter 12: Are We Living in a Computer Simulation? . . . . . . . . . . . . . . . . . . . . . . . 269 Appendix: Practice Project Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315

C O N T E N T S I N D E T A I L ACKNOWLEDGMENTS xvii INTRODUCTION xix Who Should Read This Book? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xx Why Python? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xx What’s in This Book? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xx Python Version, Platform, and IDE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxii Installing Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxii Running Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xxiv Using a Virtual Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxv Onward! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxv 1 SAVING SHIPWRECKED SAILORS WITH BAYES’ RULE 1 Bayes’ Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Project #1: Search and Rescue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 The Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Installing the Python Libraries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 The Bayes Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Playing the Game . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 Challenge Project: Smarter Searches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 Challenge Project: Finding the Best Strategy with MCS . . . . . . . . . . . . . . . . . . . . . . . . 25 Challenge Project: Calculating the Probability of Detection . . . . . . . . . . . . . . . . . . . . . 25 2 ATTRIBUTING AUTHORSHIP WITH STYLOMETRY 27 Project #2: The Hound, The War, and The Lost World . . . . . . . . . . . . . . . . . . . . . . . . 28 The Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 Installing NLTK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 The Corpora . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 The Stylometry Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 Practice Project: Hunting the Hound with Dispersion . . . . . . . . . . . . . . . . . . . . . . . . . . 48 Practice Project: Punctuation Heatmap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 Challenge Project: Fixing Frequency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

xii Contents in Detail 3 SUMMARIZING SPEECHES WITH NATURAL LANGUAGE PROCESSING 51 Project #3: I Have a Dream . . . to Summarize Speeches! . . . . . . . . . . . . . . . . . . . . . . 52 The Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 Web Scraping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 The “I Have a Dream” Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 Project #4: Summarizing Speeches with gensim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 Installing gensim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 The Make Your Bed Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 Project #5: Summarizing Text with Word Clouds . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 The Word Cloud and PIL Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 The Word Cloud Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 Fine-Tuning the Word Cloud . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 Challenge Project: Game Night . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 Challenge Project: Summarizing Summaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 Challenge Project: Summarizing a Novel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 Challenge Project: It’s Not Just What You Say, It’s How You Say It! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 4 SENDING SUPER-SECRET MESSAGES WITH A BOOK CIPHER 77 The One-Time Pad . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 The Rebecca Cipher . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 Project #6: The Digital Key to Rebecca . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 The Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 The Encryption Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 Sending Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 Practice Project: Charting the Characters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 Practice Project: Sending Secrets the WWII Way . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 5 FINDING PLUTO 95 Project #7: Replicating a Blink Comparator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 The Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 The Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 The Blink Comparator Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 Using the Blink Comparator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 Project #8: Detecting Astronomical Transients with Image Differencing . . . . . . . . . . . . 112 The Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 The Transient Detector Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 Using the Transient Detector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 Practice Project: Plotting the Orbital Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

Contents in Detail xiii Practice Project: What’s the Difference? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 Challenge Project: Counting Stars . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 6 WINNING THE MOON RACE WITH APOLLO 8 123 Understanding the Apollo 8 Mission . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 The Free Return Trajectory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 The Three-Body Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 Project #9: To the Moon with Apollo 8! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 Using the turtle Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 The Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 The Apollo 8 Free Return Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 Running the Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146 Practice Project: Simulating a Search Pattern . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146 Practice Project: Start Me Up! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 Practice Project: Shut Me Down! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148 Challenge Project: True-Scale Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 Challenge Project: The Real Apollo 8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 7 SELECTING MARTIAN LANDING SITES 151 How to Land on Mars . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152 The MOLA Map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 Project #10: Selecting Martian Landing Sites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 The Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 The Site Selector Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171 Practice Project: Confirming That Drawings Become Part of an Image . . . . . . . . . . . . . 172 Practice Project: Extracting an Elevation Profile . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172 Practice Project: Plotting in 3D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173 Practice Project: Mixing Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173 Challenge Project: Making It Three in a Row . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175 Challenge Project: Wrapping Rectangles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175 8 DETECTING DISTANT EXOPLANETS 177 Transit Photometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178 Project #11: Simulating an Exoplanet Transit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 The Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180 The Transit Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 Experimenting with Transit Photometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186 Project #12: Imaging Exoplanets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188 The Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188 The Pixelator Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

xiv Contents in Detail Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194 Practice Project: Detecting Alien Megastructures . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 Practice Project: Detecting Asteroid Transits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 Practice Project: Incorporating Limb Darkening . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198 Practice Project: Detecting Starspots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200 Practice Project: Detecting an Alien Armada . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200 Practice Project: Detecting a Planet with a Moon . . . . . . . . . . . . . . . . . . . . . . . . . . . 201 Practice Project: Measuring the Length of an Exoplanet’s Day . . . . . . . . . . . . . . . . . . 201 Challenge Project: Generating a Dynamic Light Curve . . . . . . . . . . . . . . . . . . . . . . . 202 9 IDENTIFYING FRIEND OR FOE 203 Detecting Faces in Photographs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204 Project #13: Programming a Robot Sentry Gun . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205 The Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207 The Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218 Detecting Faces from a Video Stream . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222 Practice Project: Blurring Faces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222 Challenge Project: Detecting Cat Faces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223 10 RESTRICTING ACCESS WITH FACE RECOGNITION 225 Recognizing Faces with Local Binary Pattern Histograms . . . . . . . . . . . . . . . . . . . . . . 226 The Face Recognition Flowchart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226 Extracting Local Binary Pattern Histograms . . . . . . . . . . . . . . . . . . . . . . . . . 228 Project #14: Restricting Access to the Alien Artifact . . . . . . . . . . . . . . . . . . . . . . . . . 231 The Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231 Supporting Modules and Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231 The Video Capture Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232 The Face Trainer Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236 The Face Predictor Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242 Challenge Project: Adding a Password and Video Capture . . . . . . . . . . . . . . . . . . . . 242 Challenge Project: Look-Alikes and Twins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243 Challenge Project: Time Machine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243 11 CREATING AN INTERACTIVE ZOMBIE ESCAPE MAP 245 Project #15: Visualizing Population Density with a Choropleth Map . . . . . . . . . . . . . . 246 The Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247 The Python Data Analysis Library . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248 The bokeh and holoviews Libraries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249 Installing pandas, bokeh, and holoviews . . . . . . . . . . . . . . . . . . . . . . . . . . 250

Contents in Detail xv Accessing the County, State, Unemployment, and Population Data . . . . . . . . 250 Hacking holoviews . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252 The Choropleth Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254 Planning the Escape . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266 Challenge Project: Mapping US Population Change . . . . . . . . . . . . . . . . . . . . . . . . . 266 12 ARE WE LIVING IN A COMPUTER SIMULATION? 269 Project #16: Life, the Universe, and Yertle’s Pond . . . . . . . . . . . . . . . . . . . . . . . . . . . 270 The Pond Simulation Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270 Implications of the Pond Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273 Measuring the Cost of Crossing the Lattice . . . . . . . . . . . . . . . . . . . . . . . . . 275 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277 The Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279 Moving On . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279 Challenge Project: Finding a Safe Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279 Challenge Project: Here Comes the Sun . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280 Challenge Project: Seeing Through a Dog’s Eyes . . . . . . . . . . . . . . . . . . . . . . . . . . . 281 Challenge Project: Customized Word Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281 Challenge Project: Simplifying a Celebration Slideshow . . . . . . . . . . . . . . . . . . . . . . 281 Challenge Project: What a Tangled Web We Weave . . . . . . . . . . . . . . . . . . . . . . . 281 Challenge Project: Go Tell It on the Mountain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281 APPENDIX PRACTICE PROJECT SOLUTIONS 283 Chapter 2: Attributing Authorship with Stylometry . . . . . . . . . . . . . . . . . . . . . . . . . . 283 Hunting the Hound with Dispersion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283 Punctuation Heatmap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284 Chapter 4: Sending Super-Secret Messages with a Book Cipher . . . . . . . . . . . . . . . . 285 Charting the Characters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285 Sending Secrets the WWII Way . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286 Chapter 5: Finding Pluto . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289 Plotting the Orbital Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289 What’s the Difference? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290 Chapter 6: Winning the Moon Race with Apollo 8 . . . . . . . . . . . . . . . . . . . . . . . . . . 292 Simulating a Search Pattern . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292 Start Me Up! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293 Shut Me Down! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296 Chapter 7: Selecting Martian Landing Sites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298 Confirming That Drawings Become Part of an Image . . . . . . . . . . . . . . . . . . 298 Extracting an Elevation Profile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298 Plotting in 3D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299 Mixing Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300 Chapter 8: Detecting Distant Exoplanets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304 Detecting Alien Megastructures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304 Detecting Asteroid Transits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305

xvi Contents in Detail Incorporating Limb Darkening . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306 Detecting an Alien Armada . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307 Detecting a Planet with a Moon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309 Measuring the Length of an Exoplanet’s Day . . . . . . . . . . . . . . . . . . . . . . . 311 Chapter 9: Identifying Friend or Foe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312 Blurring Faces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312 Chapter 10: Restricting Access with Face Recognition . . . . . . . . . . . . . . . . . . . . . . . 312 Challenge Project: Adding a Password and Video Capture . . . . . . . . . . . . . 312 INDEX 315

A C K N O W L E D G M E N T S Despite operating during a global pandemic, the team at No Starch Press delivered another excellent effort at book making. They are professionals without peer, and this book would not exist without them. They have my deepest gratitude and respect. Thanks also to Chris Kren and Eric Evenchick for their code reviews, Joseph B. Paul and Sarah and Lora Vaughan for their cosplay enthusiasm, and Hannah Vaughan for supplying useful photographs. Special thanks to Eric T. Mortenson for his meticulous technical reviews and many helpful suggestions and additions. Eric proposed the chapter on Bayes’ Rule and supplied numerous practice and challenge projects including applying Monte Carlo simulation to Bayes, summarizing a novel by chapter, modeling interactions between the moon and Apollo 8, viewing Mars in 3D, calculating the light curve for an exoplanet with an orbiting moon, and more. This book is immensely better for his efforts. Finally, thanks to all the contributors to stackoverflow.com. One of the best things about Python is its extensive and inclusive user community. No matter what question you may have, someone can answer it; no matter what strange thing you want to do, someone has probably done it before, and you can find them on Stack Overflow.