Nginx From Beginner to Pro (Soni, Rahul) (Z-Library)

Nginx From Beginner to Pro — Rahul Soni

Nginx From Beginner to Pro Rahul Soni

Nginx: From Beginner to Pro Rahul Soni Kolkata, West Bengal India ISBN-13 (pbk): 978-1-4842-1657-6 ISBN-13 (electronic): 978-1-4842-1656-9 DOI 10.1007/978-1-4842-1656-9 Library of Congress Control Number: 2016951451 Copyright © 2016 by Rahul Soni This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection with reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publisher’s location, in its current version, and permission for use must always be obtained from Springer. Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations are liable to prosecution under the respective Copyright Law. Trademarked names, logos, and images may appear in this book. Rather than use a trademark symbol with every occurrence of a trademarked name, logo, or image we use the names, logos, and images only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein. Managing Director: Welmoed Spahr Acquisitions Editor: Louise Corrigan Development Editor: Corbin Collins Technical Reviewer: Eduardo Balsa Editorial Board: Steve Anglin, Pramila Balen, Laura Berendson, Aaron Black, Louise Corrigan, James DeWolf, Jonathan Gennick, Todd Green, Robert Hutchinson, Celestin Suresh John, Nikhil Karkal, James Markham, Susan McDermott, Matthew Moodie, Natalie Pao, Gwenan Spearing Coordinating Editor: Nancy Chen Copy Editor: Karen Jameson Compositor: SPi Global Indexer: SPi Global Distributed to the book trade worldwide by Springer Science+Business Media New York, 233 Spring Street, 6th Floor, New York, NY 10013. Phone 1-800-SPRINGER, fax (201) 348-4505, e-mail orders-ny@springer-sbm.com , or visit www.springer.com . Apress Media, LLC is a California LLC and the sole member (owner) is Springer Science + Business Media Finance Inc (SSBM Finance Inc). SSBM Finance Inc is a Delaware corporation. For information on translations, please e-mail rights@apress.com , or visit www.apress.com . Apress and friends of ED books may be purchased in bulk for academic, corporate, or promotional use. eBook versions and licenses are also available for most titles. For more information, reference our Special Bulk Sales–eBook Licensing web page at www.apress.com/bulk-sales . Any source code or other supplementary materials referenced by the author in this text is available to readers at www.apress.com . For detailed information about how to locate your book’s source code, go to www.apress.com/ source-code/ . Printed on acid-free paper

This book is dedicated to the open source community at large who keep pushing the limits of what software can achieve!

v Contents at a Glance About the Author ...................................................................................................xvii About the Technical Reviewer ................................................................................xix Acknowledgments ..................................................................................................xxi ■Chapter 1: Introduction to Nginx Web Server ........................................................ 1 ■Chapter 2: Installing Nginx .................................................................................. 17 ■Chapter 3: Nginx Core Directives ......................................................................... 49 ■Chapter 4: Nginx Modules ................................................................................... 77 ■Chapter 5: Nginx Core Architecture ..................................................................... 97 ■Chapter 6: Hosting Web Sites on Nginx ............................................................. 107 ■Chapter 7: Nginx and Dynamic Content ............................................................. 131 ■Chapter 8: Load Balancing with Nginx .............................................................. 153 ■Chapter 9: Log Analysis, Monitoring, and Automation ...................................... 173 ■Chapter 10: SSL, Security, and Authentication .................................................. 195 ■Chapter 11: Upgrading and Migrating ............................................................... 209 ■Chapter 12: Troubleshooting Tips and FAQ ........................................................ 217 Index ..................................................................................................................... 235

vii Contents About the Author ...................................................................................................xvii About the Technical Reviewer ................................................................................xix Acknowledgments ..................................................................................................xxi ■Chapter 1: Introduction to Nginx Web Server ........................................................ 1 HTTP Basics ..................................................................................................................... 1 What Is a Web Server? ..................................................................................................... 3 Seven Reasons Why You Should Be Using Nginx ............................................................. 4 It’s Fast ................................................................................................................................................... 4 It Can Accelerate Your Application .......................................................................................................... 4 It Has a Straightforward Load Balancer .................................................................................................. 4 It Scales Well .......................................................................................................................................... 4 You Can Upgrade It On the Fly ................................................................................................................ 5 It’s Affordable to Install and Maintain ..................................................................................................... 5 It’s Easy to Use ....................................................................................................................................... 5 Main Features of Nginx .................................................................................................... 5 More Than Just a Web Server ................................................................................................................. 5 Modular Design ...................................................................................................................................... 6 Asynchronous Web Server ...................................................................................................................... 6 Reverse Proxy and Load Balancing Capability ........................................................................................ 6 Low Resource Requirement and Consumption ....................................................................................... 6 Unparalleled Performance ...................................................................................................................... 7 Multiple Protocol Support: HTTP(S), WebSocket, IMAP, POP3, SMTP ...................................................... 7 SSL Termination ...................................................................................................................................... 7 HTTP Video Streaming Using MP4/FLV/HDS/HLS .................................................................................... 7

■ CONTENTS viii Extended Monitoring and Logging .......................................................................................................... 8 Graceful Restarting ................................................................................................................................. 8 Upgrades without Downtime Using Live Binaries ................................................................................... 8 Enterprise Features of Nginx Plus .......................................................................................................... 8 Support Available with Nginx Plus .......................................................................................................... 9 Advantages of Nginx Plus ................................................................................................. 9 Advanced HTTP and TCP Load Balancing ............................................................................................... 9 Session Persistence ............................................................................................................................... 9 Content Caching Enhanced Capabilities ................................................................................................. 9 Application Health Checks .................................................................................................................... 10 HTTP Live Streaming (HLS) and Video on Demand (VOD) ..................................................................... 10 HTTP Dynamic Streaming (HDS/VOD) ................................................................................................... 11 Bandwidth Management for MP4 Media .............................................................................................. 11 Live Activity Monitoring ........................................................................................................................ 11 Nginx Commercial Support ................................................................................................................... 12 Differences between Apache and Nginx ........................................................................ 12 History .................................................................................................................................................. 13 Performance ......................................................................................................................................... 13 Resource Requirements ....................................................................................................................... 13 Availability ............................................................................................................................................ 13 Proxy and Load Balancing Server ......................................................................................................... 14 Static vs. Dynamic Content ................................................................................................................... 14 Confi guration ........................................................................................................................................ 14 Modules (or Plug-Ins) ........................................................................................................................... 14 Documentation ..................................................................................................................................... 15 Support ................................................................................................................................................. 15 Summary ........................................................................................................................ 15

■ CONTENTS ix ■Chapter 2: Installing Nginx .................................................................................. 17 Preparing Your Environment ........................................................................................... 17 Installing Nginx Using Pre-Built Packages ..................................................................... 19 Install Nginx Pre-Built Package ............................................................................................................ 19 Nginx Folder Structure .......................................................................................................................... 21 Uninstall Nginx ...................................................................................................................................... 22 Downloading Nginx from Source .................................................................................... 23 Downloading Nginx ............................................................................................................................... 23 Extracting the Nginx Archive ................................................................................................................ 23 Understanding the Nginx Source Directory .......................................................................................... 23 Installing Nginx Binaries................................................................................................. 24 Build Tools for Compilation ................................................................................................................... 24 Install Dependent Packages ................................................................................................................. 27 Compiling Nginx ............................................................................................................. 29 Understanding the ./confi gure Script ................................................................................................... 29 Compile-Time Options .......................................................................................................................... 29 Compiling and Installing Nginx ............................................................................................................. 37 Enable Nginx Service on Reboot ........................................................................................................... 41 Troubleshooting Service Start .............................................................................................................. 41 Verifying Web Server Installation ................................................................................... 42 Firewall Confi guration .......................................................................................................................... 43 Nginx in Amazon Elastic Compute Cloud (EC2) .............................................................. 44 Summary ........................................................................................................................ 47 ■Chapter 3: Nginx Core Directives ......................................................................... 49 Location of Confi guration Files ....................................................................................... 49 What Are Directives? ...................................................................................................... 49 Context Types ................................................................................................................. 50 Understanding the Default Confi guration ....................................................................... 51 Simple Directives .................................................................................................................................. 51 Events Context ...................................................................................................................................... 53

■ CONTENTS x HTTP Context ........................................................................................................................................ 54 The conf.d Folder .................................................................................................................................. 56 Server Context ...................................................................................................................................... 57 Location Context ................................................................................................................................... 62 Location Context Special Cases ............................................................................................................ 69 Verify the Correctness of Confi guration ................................................................................................ 72 Allow Directory Listing .......................................................................................................................... 72 Deny Access to Any Specifi c Location .................................................................................................. 73 Proxy the Requests to Apache .............................................................................................................. 73 Proxy the Requests to FastCGI .............................................................................................................. 73 Nginx Variables ............................................................................................................... 74 A Quick Note about Nginx Offi cial Documentation ......................................................... 75 Summary ........................................................................................................................ 75 ■Chapter 4: Nginx Modules ................................................................................... 77 What Are Modules? ........................................................................................................ 77 Module Installation ............................................................................................................................... 77 Module Categories ................................................................................................................................ 79 How Does a Module Work?............................................................................................. 80 Module Structure .................................................................................................................................. 80 How Modules Fit in Nginx ..................................................................................................................... 80 Confi guring Default Modules for Optimal Performance.................................................. 81 Core Module ......................................................................................................................................... 81 Events Module ...................................................................................................................................... 83 HTTP Module ........................................................................................................................................ 84 Enabling Optional Modules ............................................................................................. 88 Gzip Module .......................................................................................................................................... 88 FastCGI Module ..................................................................................................................................... 89 Basic Authentication ............................................................................................................................. 90

■ CONTENTS xi Enabling Third-Party Modules ........................................................................................ 91 PAM Authentication .............................................................................................................................. 91 Summary ........................................................................................................................ 95 ■Chapter 5: Nginx Core Architecture ..................................................................... 97 A Quick Analogy ............................................................................................................. 97 The Master Process ........................................................................................................ 98 Processes vs. Threads .......................................................................................................................... 99 The Worker Process ............................................................................................................................ 100 State Machines ................................................................................................................................... 102 Update Confi guration .......................................................................................................................... 104 Upgrade .............................................................................................................................................. 104 HTTP Request Processing in Nginx ..................................................................................................... 104 Summary ...................................................................................................................... 106 ■Chapter 6: Hosting Web Sites on Nginx ............................................................. 107 Web Server Setup ......................................................................................................... 108 Connecting Host and Guest Servers ................................................................................................... 110 User Creation ...................................................................................................................................... 111 Sample Applications ........................................................................................................................... 112 Uploading Content .............................................................................................................................. 112 Hosting Websites ................................................................................................................................ 115 Websites Using Different Names ........................................................................................................ 119 Websites Using Domain Name ............................................................................................................ 121 Internal Redirects ............................................................................................................................... 122 Sites Using Different Ports.................................................................................................................. 123 Wildcard Mapping ............................................................................................................................... 124 Blocking Access .................................................................................................................................. 125 Domain Name Mapping ...................................................................................................................... 125 IP-Based Hosting .......................................................................................................... 126 Mixed Name-Based and IP-Based Servers ......................................................................................... 127

■ CONTENTS xii Common Confi guration Mistakes ................................................................................. 127 Let’s Use 777 ...................................................................................................................................... 127 Root Inside Location Block ................................................................................................................. 127 Monolithic Confi guration Files ............................................................................................................ 128 Unnecessary Complications ............................................................................................................... 128 Listening on Hostname ....................................................................................................................... 129 Summary ...................................................................................................................... 129 ■Chapter 7: Nginx and Dynamic Content ............................................................. 131 Sudo Scare ................................................................................................................... 131 Installing MySQL ........................................................................................................... 132 Installing PHP ............................................................................................................... 134 Confi gure Nginx for WordPress .................................................................................... 139 The MEAN Stack ........................................................................................................... 148 Installing MongoDB............................................................................................................................. 148 Installing Node.JS ............................................................................................................................... 149 Installing Express.JS .......................................................................................................................... 149 Installing Angular.JS ........................................................................................................................... 149 Creating a MEAN Application .............................................................................................................. 150 Confi gure Nginx for MEAN Stack .................................................................................. 151 Summary ...................................................................................................................... 152 ■Chapter 8: Load Balancing with Nginx .............................................................. 153 Defi ning High Availability .............................................................................................. 153 Load Balancing for High Availability ............................................................................. 155 Hardware Load Balancer .................................................................................................................... 156 Software Load Balancer ..................................................................................................................... 156 Load Balancing in Nginx ............................................................................................... 158 Clean Up the Servers .......................................................................................................................... 158 Create Web Content ............................................................................................................................ 159 Confi gure WFE1 and WFE2 ................................................................................................................. 159 Set Up NLB Server .............................................................................................................................. 160

■ CONTENTS xiii Load Balancing Algorithms ................................................................................................................. 162 Most Suitable Algorithm ..................................................................................................................... 164 Load Balancing Scenarios ............................................................................................ 164 Nginx Routing Request to Express/Node.js ........................................................................................ 165 Passing the HOST Header ................................................................................................................... 165 Forwarding IP Information .................................................................................................................. 167 Buffering ............................................................................................................................................. 168 Nginx Caching ..................................................................................................................................... 168 Server Directive Additional Parameters .............................................................................................. 169 Confi gure Nginx (PLUS) for Heath Checks .................................................................... 169 Activity Monitoring in Nginx (PLUS) .............................................................................. 170 Summary ...................................................................................................................... 171 ■Chapter 9: Log Analysis, Monitoring, and Automation ...................................... 173 Error Log ....................................................................................................................... 173 Access Log ................................................................................................................... 175 What to Log? ....................................................................................................................................... 176 Log Buffers ......................................................................................................................................... 177 Conditional Log ................................................................................................................................... 177 Log Compression ................................................................................................................................ 177 Syslog ........................................................................................................................... 178 Analyze Logs ................................................................................................................ 179 tail ....................................................................................................................................................... 179 ngxtop ................................................................................................................................................. 179 GoAccess ............................................................................................................................................ 182 Custom Error pages ...................................................................................................... 184 Benchmark ................................................................................................................... 187 Apache Benchmark ............................................................................................................................ 188 JMeter ................................................................................................................................................ 190 Cloud-Based Benchmarking ............................................................................................................... 190

■ CONTENTS xiv Baseline ........................................................................................................................ 190 Monitoring .................................................................................................................... 190 Nginx PLUS ......................................................................................................................................... 191 Automation ................................................................................................................... 192 Summary ...................................................................................................................... 194 ■Chapter 10: SSL, Security, and Authentication .................................................. 195 Tools to Protect Network Traffi c ................................................................................... 195 Capturing Network Traffi c with Wireshark .......................................................................................... 195 Using Secure HTTP ....................................................................................................... 197 SSL Encryption ............................................................................................................. 200 Enabling SSL Nginx ............................................................................................................................ 201 Optimizing HTTPS Servers .................................................................................................................. 203 Creating a Certifi cate Request ............................................................................................................ 205 Web Server Security ..................................................................................................... 206 Creating the Password File ................................................................................................................. 206 Confi guring Nginx Password Authentication ...................................................................................... 207 Summary ...................................................................................................................... 208 ■Chapter 11: Upgrading and Migrating ............................................................... 209 Controlling Nginx .......................................................................................................... 209 Command-Line Parameters ................................................................................................................ 210 Migrating from Apache to Nginx ................................................................................... 213 Feature Comparison ........................................................................................................................... 213 Confi guration Comparison .................................................................................................................. 215 Summary ...................................................................................................................... 216 ■Chapter 12: Troubleshooting Tips and FAQ ........................................................ 217 First, What You Should Not Do ...................................................................................... 217 First Commandment of Troubleshooting: Isolate the Issue .......................................... 217 Scenario 1: Page Cannot Be Displayed in the Browser ...................................................................... 217 Scenario 2: Confl icting Ports .............................................................................................................. 220

■ CONTENTS xv Scenario 3: Bad Permissions .............................................................................................................. 220 Scenario 4: Bad Confi guration ............................................................................................................ 221 Scenario 5: Rewrite Rules .................................................................................................................. 221 Scenario 6: Log Only Your Requests ................................................................................................... 221 Important Tools for Web Administrators ....................................................................... 222 ping ..................................................................................................................................................... 222 traceroute ........................................................................................................................................... 222 top ...................................................................................................................................................... 222 htop .................................................................................................................................................... 222 atop..................................................................................................................................................... 223 uptime ................................................................................................................................................ 223 free ..................................................................................................................................................... 223 ifconfi g or ip addr ............................................................................................................................... 223 ulimit................................................................................................................................................... 223 nslookup ............................................................................................................................................. 224 powertop ............................................................................................................................................ 224 iotop .................................................................................................................................................... 225 iptraf ................................................................................................................................................... 226 tcpdump ............................................................................................................................................. 226 WireShark ........................................................................................................................................... 226 Nagios ................................................................................................................................................. 226 zabbix ................................................................................................................................................. 227 w ......................................................................................................................................................... 227 lsof ...................................................................................................................................................... 227 Common Pitfalls to Avoid.............................................................................................. 227 Chmod 777 ......................................................................................................................................... 227 Having Root Inside Location Block ..................................................................................................... 227 Using if Blocks .................................................................................................................................... 228 Passing Uncontrolled Requests to PHP .............................................................................................. 230 Rewrite Issues .................................................................................................................................... 230 Using Hostname in Confi guration ....................................................................................................... 231

■ CONTENTS xvi Frequently Asked Questions ......................................................................................... 231 “Is there an option to compare Nginx and Nginx Plus?” .................................................................... 231 “Is there a location for sample confi gurations?” ................................................................................ 231 “How can I redirect from www to no-www and vice versa?” ............................................................ 232 “How can I write all http requests to https while maintaining a sub-domain?”................................. 232 “How can I fi nd which fl ags Nginx was compiled with?” ................................................................... 233 “Is there any mechanism for detailed debugging?” ........................................................................... 233 “How many third-party modules does Nginx have?” ......................................................................... 233 “What happens if I have Nginx Plus and the license expires?” .......................................................... 233 “Is there design or consulting help available?” .................................................................................. 233 Summary ...................................................................................................................... 233 Index ..................................................................................................................... 235

xvii About the Author Rahul Soni worked at Microsoft for a decade (2004–2014) before becoming completely fascinated and then pulled into open source technologies. He consulted on IIS, .NET, and SharePoint during his Microsoft tenure. There was no looking back when he realized how much could be achieved using the open source stack. Nginx and the MEAN stack floored him completely, and he founded Attosol Technologies ( http://www.attosol.com ) to pursue his passion. This book is an honest attempt from a guy who has always been into a proprietary stack, but ultimately found that the open source community is exciting enough to leave the corporate world. Rahul is a man of simple taste and loves reading, listening to music, and traveling. When not working, he can be found hanging around with his wife Neha and their two kids Anika and Aarav.

xix About the Technical Reviewer Eduardo Balsa has worked in system administration for the last 13 years, focusing on server scaling and security. His work on the Fintech and web marketing sector has led him to push web server software to its limits and to gather an extensive knowledge of the ins and outs of their deployment, configuration, and day-to-day maintenance. Eduardo is adept at system-wide automation and of the K.I.S.S. principle and passionate about tinkering and photography. You can find him online at https://ebalsa.org .