Linux Firewalls Attack Detection and Response with IPTABLES, PSAD, and FWSNORFT (Unknown) (Z-Library)

📄 File Format: PDF

💾 File Size: 2.5 MB

📖 Read Online ⬇️ Download

Registered users can read the full content for free

Register as a Gaohf Library member to read the complete e-book online for free and enjoy a better reading experience.

📄 Page 1

www.nostarch.com TH E F I N EST I N G E E K E NTE RTA I N M E NT™ SHELVE IN: COM PUTER SECURITY/ NETW ORKING $49.95 ($59.95 CDN) ® U S E I P T A B L E S T O D E T E C T A N D P R E V E N T N E T W O R K - B A S E D A T T A C K S “ I LAY F LAT .” Th is book uses RepKover — a durable b ind ing that won’t snap shut. Printed on recycled paper System administrators need to stay ahead of new security vulnerabilities that leave their networks exposed every day. A firewall and an intrusion detection system (IDS) are two important weapons in that fight, enabling you to proactively deny access and monitor network traffic for signs of an attack. Linux Firewalls discusses the technical details of the iptables firewall and the Netfilter framework that are built into the Linux kernel, and it explains how they provide strong filtering, Network Address Translation (NAT), state tracking, and application layer inspection capabilities that rival many commercial tools. You’ll learn how to deploy iptables as an IDS with psad and fwsnort and how to build a strong, passive authentica- tion layer around iptables with fwknop. Concrete examples illustrate concepts such as firewall log analysis and policies, passive network authentica- tion and authorization, exploit packet traces, Snort ruleset emulation, and more with coverage of: • Application layer attack detection with the iptables string match extension and fwsnort • Building an iptables ruleset that emulates a Snort ruleset • Port knocking vs. Single Packet Authorization (SPA) • Tools for visualizing iptables logs • Passive OS fingerprinting with iptables Perl and C code snippets offer practical examples that will help you to maximize your deployment of Linux firewalls. If you’re responsible for keeping a network secure, you’ll find Linux Firewalls invaluable in your attempt to understand attacks and use iptables—along with psad and fwsnort—to detect and even prevent compromises. A B O U T T H E A U T H O R Michael Rash is a security architect with Enterasys Networks, Inc., where he develops the Dragon intrusion detection and prevention system. He is a frequent contributor to open source projects and the creator of psad, fwknop, and fwsnort. Rash is an expert on firewalls, intrusion detection systems, passive OS fingerprinting, and the Snort rules language. He is co-author of Snort 2.1 Intrusion Detection (Syngress, 2004) and author of Intrusion Prevention and Active Response (Syngress, 2005), and he has written security articles for Linux Journal, Sys Admin maga- zine, and ;login:. L I N U X F I R E W A L L S A T T A C K D E T E C T I O N A N D R E S P O N S E W I T H I P T A B L E S , P S A D , A N D F W S N O R T M I C H A E L R A S H ® Linux Firewalls is a great book. —From the foreword by Richard Bejtlich of TaoSecurity.com L IN U X F IR E W A L L S L IN U X F IR E W A L L S R A S H

📄 Page 2

(This page has no text content)

📄 Page 3

LINUX FIREWALLS

📄 Page 4

(This page has no text content)

📄 Page 5

LINUX FIRE WALLS A t t a c k D e t e c t i o n a n d R e s p o n s e w i t h i p t a b l e s, p s a d , a n d f w s n o r t by Michael Rash San Francisco ®

📄 Page 6

LINUX FIREWALLS. Copyright © 2007 by Michael Rash. All rights reserved. No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval system, without the prior written permission of the copyright owner and the publisher. 11 10 09 08 07 1 2 3 4 5 6 7 8 9 ISBN-10: 1-59327-141-7 ISBN-13: 978-1-59327-141-1 Publisher: William Pollock Production Editor: Christina Samuell Cover and Interior Design: Octopod Studios Developmental Editor: William Pollock Technical Reviewer: Pablo Neira Ayuso Copyeditors: Megan Dunchak and Bonnie Granat Compositors: Christina Samuell and Riley Hoffman Proofreaders: Karol Jurado and Riley Hoffman Indexer: Nancy Guenther For information on book distributors or translations, please contact No Starch Press, Inc. directly: No Starch Press, Inc. 555 De Haro Street, Suite 250, San Francisco, CA 94107 phone: 415.863.9900; fax: 415.863.9950; info@nostarch.com; www.nostarch.com Library of Congress Cataloging-in-Publication Data Rash, Michael. Linux firewalls : attack detection and response with iptables, psad, and fwsnort / Michael Rash. p. cm. Includes index. ISBN-13: 978-1-59327-141-1 ISBN-10: 1-59327-141-7 1. Computers--Access control. 2. Firewalls (Computer security) 3. Linux. I. Title. QA76.9.A25R36 2007 005.8--dc22 2006026679 No Starch Press and the No Starch Press logo are registered trademarks of No Starch Press, Inc. Other product and company names mentioned herein may be the trademarks of their respective owners. Rather than use a trademark symbol with every occurrence of a trademarked name, we are using the names only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark. The information in this book is distributed on an “As Is” basis, without warranty. While every precaution has been taken in the preparation of this work, neither the author nor No Starch Press, Inc. shall have any liability to any person or entity with respect to any loss or damage caused or alleged to be caused directly or indirectly by the information contained in it. Printed on recycled paper in the United States of America

📄 Page 7

To Katie and little Bella

📄 Page 8

(This page has no text content)

📄 Page 9

B R I E F C O N T E N T S Acknowledgments ..........................................................................................................xv Foreword by Richard Bejtlich ......................................................................................... xvii Introduction ....................................................................................................................1 Chapter 1: Care and Feeding of iptables ...........................................................................9 Chapter 2: Network Layer Attacks and Defense ................................................................35 Chapter 3: Transport Layer Attacks and Defense ...............................................................49 Chapter 4: Application Layer Attacks and Defense ............................................................69 Chapter 5: Introducing psad: The Port Scan Attack Detector ...............................................81 Chapter 6: psad Operations: Detecting Suspicious Traffic ..................................................99 Chapter 7: Advanced psad Topics: From Signature Matching to OS Fingerprinting .............113 Chapter 8: Active Response with psad...........................................................................131 Chapter 9: Translating Snort Rules into iptables Rules ......................................................149 Chapter 10: Deploying fwsnort .....................................................................................173 Chapter 11: Combining psad and fwsnort .....................................................................193 Chapter 12: Port Knocking vs. Single Packet Authorization ..............................................213 Chapter 13: Introducing fwknop ...................................................................................231 Chapter 14: Visualizing iptables Logs............................................................................257 Appendix A: Attack Spoofing .......................................................................................279 Appendix B: A Complete fwsnort Script .........................................................................285 Index .........................................................................................................................291

📄 Page 10

(This page has no text content)

📄 Page 11

C O N T E N T S I N D E T A I L ACKNOWLEDGMENTS xv FOREWORD by Richard Bejt lich xvii INTRODUCTION 1 Why Detect Attacks with iptables? .............................................................................. 2 What About Dedicated Network Intrusion Detection Systems? ........................... 3 Defense in Depth ......................................................................................... 4 Prerequisites ............................................................................................................ 4 Technical References ................................................................................................ 5 About the Website ................................................................................................... 5 Chapter Summaries .................................................................................................. 6 1 CARE AND FEEDING OF IPTABLES 9 iptables ................................................................................................................... 9 Packet Filtering with iptables .................................................................................... 10 Tables ...................................................................................................... 11 Chains ..................................................................................................... 11 Matches ................................................................................................... 12 Targets .................................................................................................... 12 Installing iptables ................................................................................................... 12 Kernel Configuration .............................................................................................. 14 Essential Netfilter Compilation Options ........................................................ 15 Finishing the Kernel Configuration ............................................................... 16 Loadable Kernel Modules vs. Built-in Compilation and Security ....................... 16 Security and Minimal Compilation ............................................................................ 17 Kernel Compilation and Installation .......................................................................... 18 Installing the iptables Userland Binaries .................................................................... 19 Default iptables Policy ............................................................................................. 20 Policy Requirements ................................................................................... 20 iptables.sh Script Preamble ......................................................................... 22 The INPUT Chain ...................................................................................... 22 The OUTPUT Chain ................................................................................... 24 The FORWARD Chain ............................................................................... 25 Network Address Translation ..................................................................... 26 Activating the Policy .................................................................................. 27 iptables-save and iptables-restore ................................................................ 27 Testing the Policy: TCP ............................................................................... 29 Testing the Policy: UDP .............................................................................. 31 Testing the Policy: ICMP ............................................................................. 32 Concluding Thoughts .............................................................................................. 33

📄 Page 12

x Conten ts in Detai l 2 NETWORK LAYER ATTACKS AND DEFENSE 35 Logging Network Layer Headers with iptables ........................................................... 35 Logging the IP Header ............................................................................... 36 Network Layer Attack Definitions .............................................................................. 38 Abusing the Network Layer ..................................................................................... 39 Nmap ICMP Ping ...................................................................................... 39 IP Spoofing ............................................................................................... 40 IP Fragmentation ....................................................................................... 41 Low TTL Values .......................................................................................... 42 The Smurf Attack ....................................................................................... 43 DDoS Attacks ............................................................................................ 44 Linux Kernel IGMP Attack ........................................................................... 44 Network Layer Responses ........................................................................................ 45 Network Layer Filtering Response ................................................................ 45 Network Layer Thresholding Response ......................................................... 45 Combining Responses Across Layers ............................................................ 46 3 TRANSPORT LAYER ATTACKS AND DEFENSE 49 Logging Transport Layer Headers with iptables .......................................................... 50 Logging the TCP Header ............................................................................ 50 Logging the UDP Header ............................................................................ 52 Transport Layer Attack Definitions ............................................................................. 52 Abusing the Transport Layer .................................................................................... 53 Port Scans ................................................................................................ 53 Port Sweeps ............................................................................................. 61 TCP Sequence Prediction Attacks ................................................................. 61 SYN Floods .............................................................................................. 62 Transport Layer Responses ....................................................................................... 62 TCP Responses .......................................................................................... 62 UDP Responses ......................................................................................... 66 Firewall Rules and Router ACLs ................................................................... 67 4 APPLICATION LAYER ATTACKS AND DEFENSE 69 Application Layer String Matching with iptables ......................................................... 70 Observing the String Match Extension in Action ............................................ 70 Matching Non-Printable Application Layer Data ............................................ 71 Application Layer Attack Definitions .......................................................................... 72 Abusing the Application Layer ................................................................................. 73 Snort Signatures ........................................................................................ 74 Buffer Overflow Exploits ............................................................................. 74 SQL Injection Attacks ................................................................................. 76 Gray Matter Hacking ................................................................................. 77 Encryption and Application Encodings ...................................................................... 79 Application Layer Responses .................................................................................... 80

📄 Page 13

Conten ts in Detai l xi 5 INTRODUCING PSAD: THE PORT SCAN ATTACK DETECTOR 81 History .................................................................................................................. 81 Why Analyze Firewall Logs? ................................................................................... 82 psad Features ........................................................................................................ 83 psad Installation ..................................................................................................... 83 psad Administration ............................................................................................... 85 Starting and Stopping psad ........................................................................ 85 Daemon Process Uniqueness ...................................................................... 86 iptables Policy Configuration ...................................................................... 86 syslog Configuration .................................................................................. 88 whois Client ............................................................................................. 89 psad Configuration ................................................................................................ 90 /etc/psad/psad.conf ................................................................................ 90 /etc/psad/auto_dl .................................................................................... 96 /etc/psad/signatures ................................................................................ 96 /etc/psad/snort_rule_dl ............................................................................ 97 /etc/psad/ip_options ................................................................................ 97 /etc/psad/pf.os ....................................................................................... 97 Concluding Thoughts .............................................................................................. 98 6 PSAD OPERATIONS: DETECTING SUSPICIOUS TRAFFIC 99 Port Scan Detection with psad ................................................................................ 100 TCP connect() Scan .................................................................................. 101 TCP SYN or Half-Open Scan .................................................................... 103 TCP FIN, XMAS, and NULL Scans ............................................................. 105 UDP Scan ............................................................................................... 106 Alerts and Reporting with psad .............................................................................. 108 psad Email Alerts .................................................................................... 108 psad syslog Reporting .............................................................................. 110 Concluding Thoughts ............................................................................................ 112 7 ADVANCED PSAD TOPICS: FROM SIGNATURE MATCHING TO OS FINGERPRINTING 113 Attack Detection with Snort Rules ............................................................................ 113 Detecting the ipEye Port Scanner ............................................................... 115 Detecting the LAND Attack ....................................................................... 116 Detecting TCP Port 0 Traffic ...................................................................... 116 Detecting Zero TTL Traffic ......................................................................... 117 Detecting the Naptha Denial of Service Attack ............................................ 117 Detecting Source Routing Attempts ............................................................ 118 Detecting Windows Messenger Pop-up Spam ............................................. 118 psad Signature Updates ........................................................................................ 119 OS Fingerprinting ................................................................................................ 120 Active OS Fingerprinting with Nmap ......................................................... 120 Passive OS Fingerprinting with p0f ............................................................ 121

📄 Page 14

xii Content s i n De ta i l DShield Reporting ................................................................................................ 123 DShield Reporting Format ......................................................................... 124 Sample DShield Report ............................................................................ 124 Viewing psad Status Output .................................................................................. 124 Forensics Mode ................................................................................................... 128 Verbose/Debug Mode .......................................................................................... 128 Concluding Thoughts ............................................................................................ 130 8 ACTIVE RESPONSE WITH PSAD 131 Intrusion Prevention vs. Active Response .................................................................. 131 Active Response Trade-offs .................................................................................... 133 Classes of Attacks .................................................................................. 133 False Positives ......................................................................................... 134 Responding to Attacks with psad ............................................................................ 134 Features ................................................................................................. 135 Configuration Variables ........................................................................... 135 Active Response Examples ..................................................................................... 137 Active Response Configuration Settings ...................................................... 138 SYN Scan Response ................................................................................ 139 UDP Scan Response ................................................................................ 140 Nmap Version Scan ................................................................................ 141 FIN Scan Response .................................................................................. 141 Maliciously Spoofing a Scan .................................................................... 142 Integrating psad Active Response with Third-Party Tools ............................................ 143 Command-Line Interface ........................................................................... 143 Integrating with Swatch ............................................................................ 145 Integrating with Custom Scripts ................................................................. 146 Concluding Thoughts ............................................................................................ 147 9 TRANSLATING SNORT RULES INTO IPTABLES RULES 149 Why Run fwsnort? ................................................................................................ 150 Defense in Depth ..................................................................................... 151 Target-Based Intrusion Detection and Network Layer Defragmentation ........... 151 Lightweight Footprint ................................................................................ 152 Inline Responses ...................................................................................... 152 Signature Translation Examples .............................................................................. 153 Nmap command attempt Signature ........................................................... 153 Bleeding Snort “Bancos Trojan” Signature .................................................. 154 PGPNet connection attempt Signature ........................................................ 154 The fwsnort Interpretation of Snort Rules .................................................................. 155 Translating the Snort Rule Header .............................................................. 155 Translating Snort Rule Options: iptables Packet Logging ............................... 157 Snort Options and iptables Packet Filtering ................................................. 160 Unsupported Snort Rule Options ................................................................ 171 Concluding Thoughts ............................................................................................ 172

📄 Page 15

Conten t s in Detai l xiii 10 DEPLOYING FWSNORT 173 Installing fwsnort .................................................................................................. 173 Running fwsnort ................................................................................................... 175 Configuration File for fwsnort ................................................................... 177 Structure of fwsnort.sh .............................................................................. 179 Command-Line Options for fwsnort ........................................................... 182 Observing fwsnort in Action .................................................................................. 184 Detecting the Trin00 DDoS Tool ................................................................ 184 Detecting Linux Shellcode Traffic ............................................................... 185 Detecting and Reacting to the Dumador Trojan ........................................... 186 Detecting and Reacting to a DNS Cache-Poisoning Attack ............................ 188 Setting Up Whitelists and Blacklists ......................................................................... 191 Concluding Thoughts ............................................................................................ 192 11 COMBINING PSAD AND FWSNORT 193 Tying fwsnort Detection to psad Operations ............................................................. 194 WEB-PHP Setup.php access Attack ............................................................ 194 Revisiting Active Response ..................................................................................... 198 psad vs. fwsnort ..................................................................................... 198 Restricting psad Responses to Attacks Detected by fwsnort ............................ 199 Combining fwsnort and psad Responses .................................................... 199 DROP vs. REJECT Targets ......................................................................... 201 Thwarting Metasploit Updates ................................................................................ 204 Metasploit Update Feature ....................................................................... 204 Signature Development ............................................................................ 206 Busting Metasploit Updates with fwsnort and psad ...................................... 208 Concluding Thoughts ............................................................................................ 212 12 PORT KNOCKING VS. SINGLE PACKET AUTHORIZATION 213 Reducing the Attack Surface .................................................................................. 213 The Zero-Day Attack Problem ................................................................................. 214 Zero-Day Attack Discovery ....................................................................... 215 Implications for Signature-Based Intrusion Detection ..................................... 215 Defense in Depth ..................................................................................... 216 Port Knocking ...................................................................................................... 217 Thwarting Nmap and the Target Identification Phase ................................... 218 Shared Port-Knocking Sequences ............................................................... 218 Encrypted Port-Knocking Sequences ........................................................... 221 Architectural Limitations of Port Knocking ................................................... 223 Single Packet Authorization .................................................................................. 226 Addressing Limitations of Port Knocking ..................................................... 227 Architectural Limitations of SPA ................................................................. 228 Security Through Obscurity? .................................................................................. 229 Concluding Thoughts ............................................................................................ 230

📄 Page 16

xiv Content s i n De ta i l 13 INTRODUCING FWKNOP 231 fwknop Installation ............................................................................................... 232 fwknop Configuration ........................................................................................... 234 /etc/fwknop/fwknop.conf ....................................................................... 234 /etc/fwknop/access.conf ......................................................................... 237 Example /etc/fwknop/access.conf File ...................................................... 240 fwknop SPA Packet Format .................................................................................... 241 Deploying fwknop ................................................................................................ 243 SPA via Symmetric Encryption ................................................................... 244 SPA via Asymmetric Encryption ................................................................. 246 Detecting and Stopping a Replay Attack .................................................... 249 Spoofing the SPA Packet Source Address ................................................... 251 fwknop OpenSSH Integration Patch ........................................................... 252 SPA over Tor .......................................................................................... 254 Concluding Thoughts ............................................................................................ 255 14 VISUALIZING IPTABLES LOGS 257 Seeing the Unusual ............................................................................................... 258 Gnuplot .............................................................................................................. 260 Gnuplot Graphing Directives .................................................................... 260 Combining psad and Gnuplot .................................................................. 261 AfterGlow ........................................................................................................... 262 iptables Attack Visualizations ................................................................................. 263 Port Scans .............................................................................................. 264 Port Sweeps ........................................................................................... 267 Slammer Worm ...................................................................................... 270 Nachi Worm .......................................................................................... 272 Outbound Connections from Compromised Systems .................................... 273 Concluding Thoughts ............................................................................................ 277 A ATTACK SPOOFING 279 Connection Tracking ............................................................................................. 280 Spoofing exploit.rules Traffic .................................................................... 282 Spoofed UDP Attacks ............................................................................... 283 B A COMPLETE FWSNORT SCRIPT 285 INDEX 291

📄 Page 17

A C K N O W L E D G M E N T S Linux Firewalls was made possible with the help of a host of folks at every step along the way. I’d particularly like to thank the people at No Starch Press for the efforts they put forth. William Pollock, Bonnie Granat, Megan Dunchak, and Christina Samuell all contributed many hours of expert editing, and the book is higher quality as a result. To Pablo Neira Ayuso, thanks for helping to make Netfilter and iptables what they are today, and for handling the technical edit of the material in this book. Ron Gula, CTO of Tenable Network Security, and Raffael Marty, chief security strategist of Splunk, both contributed constructive criticism, and they were kind enough to endorse the book before it was published. I also wish to thank Richard Bejtlich, founder of TaoSecurity, for writing an excellent foreword. Richard, your books are an inspiration. My parents, James and Billie Mae, and my brother, Brian, all deserve a special thank you for their constant encouragement. Finally, many thanks go to my wife, Katie. This book would not have been possible without you.

📄 Page 18

(This page has no text content)

📄 Page 19

F O R E W O R D When hearing the term firewall, most people think of a product that inspects network traffic at the network and transport layers of the OSI Reference Model and makes pass or filter decisions. In terms of products, dozens of firewall types exist. They are differentiated by the data source they inspect (e.g., network traffic, host processes, or system calls) and the depth to which they inspect those sources. Almost any device that inspects communi- cation and decides whether to pass or filter it could be considered a firewall product. Marcus Ranum, inventor of the proxy firewall and the implementer of the first commercial firewall product, offered a definition of the term firewall in the mid-1990s when he said, “A firewall is the implementation of your Internet security policy.” 1 This is an excellent definition because it is product- neutral, timeless, and realistic. It applies equally well to the original firewall book, Firewalls and Internet Security by William R. Cheswick and Steven M. Bellovin (Addison-Wesley Professional, 1994), as it does to the book you’re reading now. 1 Computer Security Journal, Vol. XI, No. 1, Spring 1995 (http://www.spirit.com/CSI/Papers/ hownot.htm)

📄 Page 20

xviii Forewo rd In the spirit of Ranum’s definition, a firewall could also be considered a policy enforcement system. Devices that inspect and then pass or filter network traffic could be called network policy enforcement systems. Devices that inspect and then pass or filter host-centric activities could be called host policy enforce- ment systems. In either case, emphasis on policy enforcement focuses attention on the proper role of the firewall as a device that implements policy instead of one that just “stops bad stuff.” With respect to “bad stuff,” it’s reasonable to ask if firewalls even matter in today’s enterprise. Properly configured traditional network firewall pro- ducts basically deny all but allowed Internet protocols, IP addresses, TCP/UDP ports, and ICMP types and codes. In the modern attack environment, this sort of defense is entirely insufficient. Restricting those exploitation channels is necessary to restrict the ingress and egress paths to a target, but network and transport layer filtering has been a completely inadequate counter- measure for at least a decade. In 2007, the most effective way to compromise a client is to entice the user to activate a malicious executable, send the user a link that hosts malicious content, or attack another client-side component of the user’s computing experience. In many cases, exploitation doesn’t rely on a vulnerability that could be patched or a configuration that could be tightened. Rather, attackers exploit weaknesses in rich-media platforms like JavaScript and Flash, which are increasingly required for browsing the Web today. In 2007, the most effective way to compromise a server is to avoid the operating system and exploit the application. Web applications dominate the server landscape, and they are more likely to suffer from architectural and design flaws than from vulnerabilities that can be patched. In the late 1990s, it was fashionable to change the prices for the items in one’s shopping cart to demonstrate insecure web applications. Thanks to Ajax, almost a decade later the shopping cart is running on the client and users are again changing prices—and worse. All of this makes the picture seem fairly bleak for firewall products. Many have adapted by incorporating deep packet inspection or operating at or beyond the application layer of the OSI Reference Model. Others operate as intrusion prevention systems, using a clever marketing term to differentiate themselves in a seemingly commoditized market. Is there a role for firewalls, especially open source products, in the age of client-side attacks and web application exploitation? The answer is yes—and you are reading one approach right now. Michael Rash is a pioneer in the creative use of network technologies for defensive purposes. The security research and development world tends to be dominated by offensive tools and techniques, as a quick glance at the speakers list for a certain Las Vegas hacker convention will demonstrate. Bucking this trend, Michael continues to invent and improve upon methods for protecting assets from attack. After getting a look at the dark side at an offensive conference, almost all of us return to the seemingly mundane job of protecting our enterprises. Thanks to this book, we have an additional suite of programs and methods to make our jobs easier.