Professional Linux Programming (Jon Masters, Richard Blum) (Z-Library)

Professional Linux® Programming Jon Masters Richard Blum 01_776130 ffirs.qxp 2/2/07 10:13 PM Page iii

01_776130 ffirs.qxp 2/2/07 10:13 PM Page ii

Professional Linux® Programming 01_776130 ffirs.qxp 2/2/07 10:13 PM Page i

01_776130 ffirs.qxp 2/2/07 10:13 PM Page ii

Professional Linux® Programming Jon Masters Richard Blum 01_776130 ffirs.qxp 2/2/07 10:13 PM Page iii

Professional Linux® Programming Published by Wiley Publishing, Inc. 10475 Crosspoint Boulevard Indianapolis, IN 46256 www.wiley.com Copyright © 2007 by Wiley Publishing, Inc., Indianapolis, Indiana Published simultaneously in Canada Manufactured in the United States of America 10 9 8 7 6 5 4 3 2 1 Library of Congress Cataloging-in-Publication Data: Masters, Jon, 1981- Professional Linux programming / Jon Masters, Richard Blum. p. cm. Includes index. ISBN: 978-0-471-77613-0 (paper/website) 1. Linux. 2. Operating systems (Computers) I. Blum, Richard, 1962- II. Title. QA76.76.O63M37153 2007 005.4’32—dc22 2006102202 No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 646-8600. Requests to the Publisher for permission should be addressed to the Legal Department, Wiley Publishing, Inc., 10475 Crosspoint Blvd., Indianapolis, IN 46256, (317) 572-3447, fax (317) 572-4355, or online at http://www.wiley.com/go/permissions. LIMIT OF LIABILITY/DISCLAIMER OF WARRANTY: THE PUBLISHER AND THE AUTHOR MAKE NO REP- RESENTATIONS OR WARRANTIES WITH RESPECT TO THE ACCURACY OR COMPLETENESS OF THE CON- TENTS OF THIS WORK AND SPECIFICALLY DISCLAIM ALL WARRANTIES, INCLUDING WITHOUT LIMITATION WARRANTIES OF FITNESS FOR A PARTICULAR PURPOSE. NO WARRANTY MAY BE CREATED OR EXTENDED BY SALES OR PROMOTIONAL MATERIALS. THE ADVICE AND STRATEGIES CONTAINED HEREIN MAY NOT BE SUITABLE FOR EVERY SITUATION. THIS WORK IS SOLD WITH THE UNDERSTAND- ING THAT THE PUBLISHER IS NOT ENGAGED IN RENDERING LEGAL, ACCOUNTING, OR OTHER PRO- FESSIONAL SERVICES. IF PROFESSIONAL ASSISTANCE IS REQUIRED, THE SERVICES OF A COMPETENT PROFESSIONAL PERSON SHOULD BE SOUGHT. NEITHER THE PUBLISHER NOR THE AUTHOR SHALL BE LIABLE FOR DAMAGES ARISING HEREFROM. THE FACT THAT AN ORGANIZATION OR WEBSITE IS REFERRED TO IN THIS WORK AS A CITATION AND/OR A POTENTIAL SOURCE OF FURTHER INFORMA- TION DOES NOT MEAN THAT THE AUTHOR OR THE PUBLISHER ENDORSES THE INFORMATION THE ORGANIZATION OR WEBSITE MAY PROVIDE OR RECOMMENDATIONS IT MAY MAKE. FURTHER, READ- ERS SHOULD BE AWARE THAT INTERNET WEBSITES LISTED IN THIS WORK MAY HAVE CHANGED OR DISAPPEARED BETWEEN WHEN THIS WORK WAS WRITTEN AND WHEN IT IS READ. For general information on our other products and services please contact our Customer Care Department within the United States at (800) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002. Trademarks: Wiley, the Wiley logo, Wrox, the Wrox logo, Programmer to Programmer, and related trade dress are trademarks or registered trademarks of John Wiley & Sons, Inc. and/or its affiliates, in the United States and other countries, and may not be used without written permission. Linux is a registered trademark of Linus Torvalds. All other trademarks are the property of their respective owners. Wiley Publishing, Inc., is not associated with any product or vendor mentioned in this book. Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books. 01_776130 ffirs.qxp 2/2/07 10:13 PM Page iv

For Karin, whom I love very much.—Jon Masters To my wife Barbara.—Richard Blum 01_776130 ffirs.qxp 2/2/07 10:13 PM Page v

About the Authors Jon Masters is a 25-year-old British-born Linux kernel engineer, embedded systems specialist, and author who lives and works in the United States for Red Hat. Jon made UK history by becoming one of the youngest University students the country had ever seen, at the tender age of just 13. Having been through college twice by the time his peers were completing their first time around, and having been published over 100 times in a wide range of technical magazines, journals and books, Jon went on to work for a variety of multinational technology companies. He has worked extensively in the field of Embedded Systems, Enterprise Linux and Scientific instrumentation and has helped design anything and everything from Set Top Boxes to future NMR (MRI) imaging platforms. When not working on Enterprise Linux software for Red Hat, Jon likes to drink tea on Boston Common and read the collective works of Thomas Paine and other great American Revolutionaries of a bygone age. He dreams of a time when the world was driven not by electrons, but by wooden sailing ships and a universal struggle for the birth of modern nations. He plays the violin, and occasionally sings in choral ensembles, for which he has won several awards. For relaxation, Jon enjoys engaging in a little rock climbing. He lives in Cambridge, Massachusetts, just across the river Charles from historic Boston, and enjoys every minute of it. Jon has extensive experience in speaking about and training people to use a wide variety of Linux technologies and enjoys actively participating in many Linux User Groups the world over. Richard Blum has worked for over 18 years for a large U.S. government organization as a network and systems administrator. During this time he has had plenty of opportunities to work with Microsoft, Novell, and of course, UNIX and Linux servers. He has written applications and utilities using C, C++, Java, C#, Visual Basic, and shell script. Rich has a Bachelors of Science degree in Electrical Engineering, and a Masters of Science degree in Management, specializing in Management Information Systems, from Purdue University. He is the author of several books, including “sendmail for Linux” (2000, Sams publishing), “Running qmail” (2000, Sams publishing), “Postfix” (2001, Sams Publishing), “Open Source E-mail Security” (2001, Sams Publishing), “C# Network Programming” (2002, Sybex), “Network Performance Open Source Toolkit” (2003, John Wiley & Sons), and “Professional Assembly Language Programming” (2005, Wrox). When he is not being a computer nerd, Rich plays electric bass for the church worship and praise band, and enjoys spending time with his wife Barbara, and daughters Katie Jane and Jessica. 01_776130 ffirs.qxp 2/2/07 10:13 PM Page vi

Credits Contributing Writers Christopher Aillon Katherine and David Goodwin Matthew Walton Acquisitions Editor Kit Kemper Development Editor Howard A. Jones Production Editor Eric Charbonneau Copy Editor Foxxe Editorial Editorial Manager Mary Beth Wakefield Production Manager Tim Tate Vice President and Executive Group Publisher Richard Swadley Vice President and Executive Publisher Joseph B. Wikert Graphics and Production Specialists Carrie A. Foster Jennifer Mayberry Barbara Moore Alicia B. South Quality Control Technicians Cynthia Fields John Greenough Project Coordinator Adrienne Martinez Proofreading and Indexing Techbooks Anniversary Logo Design Richard Pacifico 01_776130 ffirs.qxp 2/2/07 10:13 PM Page vii

01_776130 ffirs.qxp 2/2/07 10:13 PM Page viii

Acknowledgments I sit here writing these acknowledgements on my 25th birthday, having spent many long evenings over the last year pouring over schedules, planning and even occasionally actually getting some writing done. When I first undertook to write this book, I could never have fully appreciated the amount of work it takes to put such a thing together nor the difficulties that would need to be overcome along the way. I started writing this book living just outside London and finished it less than a year later from my new home in Cambridge, Massachusetts, having decided to leave the country in the interim. Over the last year, a lot has changed for me both personally and professionally, but I am supported by a great net- work of friends and family who have helped make it possible. First and foremost I would like to thank the team I have worked with at Wiley — Debra, Adaobi, Kit, Howard and Carol as well as numerous others whose job it is to turn this manuscript into a finished book. Kit Kemper deserves special thanks for enduring my writing schedule and somehow making that just about work out in the end, as does Debra Williams-Cauley for believing that this project was a good idea in the first place. Howard Jones helped to keep me honest by doing an excellent job as my editor. This book would not exist without the inspiration I received from my good friends (and former bosses) Malcolm Buckingham and Jamie McKendry at Resonance Instruments (later Oxford Instruments), who used to moan about the lack of Linux-specific programming books. This book would also not exist with- out the kind contributions from several good friends of mine — Kat and David Goodwin, Matthew Walton, and Chris Aillon, thank you. Thanks also to Richard Blum for stepping up and joining the team once it became apparent to me that I couldn’t hope to finish this in time. You’ve all done a great job and I really do thank you very much. I have been helped along the way by my fantastic family — my parents Paula and Charles, my sisters Hannah Wrigley and Holly, my brother-in-law Joe, and occasional inspiration too from my grandmothers. I have also benefited from some of the best friends anyone could ask for — there are too many to list every- one individually, but I would like to specifically mention Hussein Jodiyawalla, Johannes Kling, Ben Swan, Paul Sladen, Markus Kobler, Tom Hawley, Sidarshan Guru Ratnavellu, Chris and Mad Ball (and Zoe, the cat), Emma Maule, John and Jan Buckman, Toby Jaffey and Sara, Sven Thorsten-Dietrich, Bill Weinberg, Daniel James, Joe Casad and Andrew Hutton and Emilie. Special thanks also to all of my friends at Red Hat, my boss and all the other hard-working people who help to make our company truly the greatest place to work anywhere in the world. Red Hat really understands what it means to work on Linux, and I am extremely grateful for having such a cool work environment, which really does encourage involve- ment in projects such as this one, in the true spirit of the Linux community — thanks, guys, you rock. Finally, I would like to thank Karin Worley for her friendship, which provided me with ample opportu- nity for procrastination during the final stages of this project. Karin, I’m not sure I would have completed it without the new-found sense of happiness that recently entered into my life. Jon Masters Cambridge, Massachusetts 01_776130 ffirs.qxp 2/2/07 10:13 PM Page ix

x Acknowledgments Many thanks go to the great team of people at Wiley for their outstanding work on this project. Thanks to Kit Kemper, the Acquisitions Editor, for offering me the opportunity to work on this book. Also thanks to Howard Jones, the Developmental Editor, for keeping things on track and helping make this book presentable. I would also like to thank Carole McClendon at Waterside Productions, Inc. for arranging this opportunity for me, and for helping out in my writing career. Finally, I would like to thank my parents, Mike and Joyce Blum, for their dedication and support while raising me, and to my wife Barbara and daughters Katie Jane and Jessica for their love, patience, and understanding, especially while I’m writing. Richard Blum 01_776130 ffirs.qxp 2/2/07 10:13 PM Page x

Contents Acknowledgments ix Introduction xix Chapter 1: Working with Linux 1 A Brief History of Linux 2 The GNU Project 2 The Linux Kernel 3 Linux Distributions 4 Free Software vs. Open Source 4 Beginning Development 5 Choosing a Linux Distribution 6 Installing a Linux Distribution 8 Linux Community 15 Linux User Groups 15 Mailing lists 16 IRC 16 Private Communities 16 Key Differences 16 Linux Is Modular 17 Linux Is Portable 17 Linux Is Generic 17 Summary 18 Chapter 2: Toolchains 19 The Linux Development Process 19 Working with Sources 20 Configuring to the Local Environment 21 Building the Sources 22 Components of the GNU Toolchain 23 The GNU Compiler Collection 23 The GNU binutils 34 GNU Make 39 The GNU Debugger 40 02_776130 ftoc.qxp 2/2/07 10:13 PM Page xi

xii Contents The Linux Kernel and the GNU Toolchain 44 Inline Assembly 44 Attribute Tags 45 Custom Linker Scripts 45 Cross-Compilation 46 Building the GNU Toolchain 47 Summary 48 Chapter 3: Portability 49 The Need for Portability 50 The Portability of Linux 51 Layers of Abstraction 51 Linux Distributions 52 Building Packages 57 Portable Source Code 70 Internationalization 81 Hardware Portability 88 64-Bit Cleanliness 89 Endian Neutrality 89 Summary 92 Chapter 4: Software Configuration Management 93 The Need for SCM 94 Centralized vs. Decentralized Development 95 Centralized Tools 95 The Concurrent Version System 96 Subversion 104 Decentralized tools 108 Bazaar-NG 109 Linux kernel SCM (git) 112 Integrated SCM Tools 115 Eclipse 115 Summary 117 Chapter 5: Network Programming 119 Linux Socket Programming 119 Sockets 120 Network Addresses 122 Using Connection-Oriented Sockets 123 Using Connectionless Sockets 130 02_776130 ftoc.qxp 2/2/07 10:13 PM Page xii

xiii Contents Moving Data 133 Datagrams vs. Streams 133 Marking Message Boundaries 137 Using Network Programming Libraries 140 The libCurl Library 140 Using the libCurl Library 141 Summary 147 Chapter 6: Databases 149 Persistent Data Storage 149 Using a Standard File 150 Using a Database 150 The Berkeley DB Package 152 Downloading and Installing 153 Building Programs 154 Basic Data Handling 154 The PostgreSQL Database Server 165 Downloading and Installing 165 Building Programs 167 Creating an Application Database 167 Connecting to the Server 169 Executing SQL Commands 173 Using Parameters 181 Summary 184 Chapter 7: Kernel Development 185 Starting Out 185 Kernel Concepts 199 A Word of Warning 200 The Task Abstraction 200 Virtual Memory 205 Don’t Panic! 208 Kernel Hacking 208 Loadable Modules 209 Kernel Development Process 211 Git: the “Stupid Content Tracker” 212 The Linux Kernel Mailing List 213 The “mm” Development Tree 215 The Stable Kernel Team 215 LWN: Linux Weekly News 216 Summary 216 02_776130 ftoc.qxp 2/2/07 10:13 PM Page xiii

xiv Contents Chapter 8: Kernel Interfaces 217 What Is an Interface? 217 Undefined Interfaces 218 External Kernel Interfaces 219 System Calls 219 The Device File Abstraction 224 Kernel Events 238 Ignoring Kernel Protections 239 Internal Kernel Interfaces 243 The Kernel API 243 The kernel ABI 244 Summary 245 Chapter 9: Linux Kernel Modules 247 How Modules Work 247 Extending the Kernel Namespace 250 No Guaranteed Module Compatibility 251 Finding Good Documentation 251 Linux Kernel Man Pages 251 Writing Linux Kernel Modules 252 Before You Begin 253 Essential Module Requirements 253 Logging 256 Exported Symbols 257 Allocating Memory 259 Locking considerations 267 Deferring work 275 Further Reading 283 Distributing Linux Kernel Modules 284 Going Upstream 284 Shipping Sources 284 Shipping Prebuilt Modules 284 Summary 285 Chapter 10: Debugging 287 Debugging Overview 287 A Word about Memory Management 288 Essential Debugging Tools 289 The GNU Debugger 289 Valgrind 298 02_776130 ftoc.qxp 2/2/07 10:13 PM Page xiv

xv Contents Graphical Debugging Tools 299 DDD 299 Eclipse 302 Kernel Debugging 305 Don’t Panic! 306 Making Sense of an oops 307 Using UML for Debugging 309 An Anecdotal Word 312 A Note about In-Kernel Debuggers 313 Summary 313 Chapter 11: The GNOME Developer Platform 315 GNOME Libraries 316 Glib 316 GObject 316 Cairo 316 GDK 317 Pango 317 GTK+ 317 libglade 318 GConf 318 GStreamer 318 Building a Music Player 319 Requirements 319 Getting Started: The Main Window 319 Building the GUI 321 Summary 340 Chapter 12: The FreeDesktop Project 341 D-BUS: The Desktop Bus 341 What Is D-Bus? 342 Under D-Hood of D-Bus 342 D-Bus Methods 346 Hardware Abstraction Layer 350 Making Hardware Just Work 350 Hal Device Objects 353 The Network Manager 358 Other Freedesktop Projects 360 Summary 360 02_776130 ftoc.qxp 2/2/07 10:13 PM Page xv

xvi Contents Chapter 13: Graphics and Audio 361 Linux and Graphics 361 X Windows 362 Open Graphics Library 364 OpenGL Utilities Toolkit 365 Simple Directmedia Layer 365 Writing OpenGL Applications 365 Downloading and Installing 366 Programming Environment 367 Using the GLUT Library 368 Writing SDL Applications 382 Downloading and Installing 382 Programming Environment 383 Using the SDL Library 383 Summary 394 Chapter 14: LAMP 395 What Is LAMP? 395 Apache 396 MySQL 396 PHP 397 The Rebel Platform 397 Evaluating the LAMP Platform 397 Apache 399 Virtual Hosting 400 Installation and Configuration of PHP 5 401 Apache Basic Authentication 402 Apache and SSL 402 Integrating SSL with HTTP Authentication 403 MySQL 404 Installing MySQL 404 Configuring and Starting the Database 404 Changing the Default Password 405 The MySQL Client Interface 405 Relational Databases 405 SQL 406 The Relational Model 409 PHP 411 The PHP Language 411 Error Handling 420 Error-Handling Exceptions 421 02_776130 ftoc.qxp 2/2/07 10:13 PM Page xvi

xvii Contents Optimization Techniques 422 Installing Additional PHP Software 427 Logging 427 Parameter Handling 428 Session Handling 429 Unit Testing 430 Databases and PHP 432 PHP Frameworks 432 The DVD Library 433 Version 1: The Developer’s Nightmare 433 Version 2: Basic Application with DB-Specific Data Layer 434 Version 3: Rewriting the Data Layer, Adding Logging and Exceptions 437 Version 4: Applying a Templating Framework 441 Summary 442 Index 443 GNU 473 02_776130 ftoc.qxp 2/2/07 10:13 PM Page xvii