RESTful Web APIs (Leonard Richardson, Mike Amundsen, Sam Ruby) (Z-Library)

Praise for RESTful Web APIs “This book is the best place to start learning the essential craft of API Design.” —Matt McLarty Cofounder, API Academy “The entire time I read this book, I was cursing. I was cursing because as I read each explanation, I was worried that they were so good that it would be hard to find a better one to use in my own writing. You will not find another work that explores the topic so thoroughly yet explains the topic so clearly. Please, take these tools, build something fantastic, and share it with the rest of the world, okay?” —Steve Klabnik Author, Designing Hypermedia APIs “Wonderfully thorough treatment of hypermedia formats, REST’s least well understood tenet." —Stefan Tilkov REST evangelist, author, and consultant “The best practical guide to hypermedia APIs. A must-have.” — Ruben Verborgh Semantic hypermedia researcher

Leonard Richardson and Mike Amundsen Foreword by Sam Ruby RESTful Web APIs

RESTful Web APIs by Leonard Richardson and Mike Amundsen with a Foreword by Sam Ruby Copyright © 2013 Leonard Richardson, amundsen.com, Inc., and Sam Ruby. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://my.safaribooksonline.com). For more information, contact our corporate/ institutional sales department: 800-998-9938 or corporate@oreilly.com. Editors: Simon St. Laurent and Meghan Blanchette Production Editor: Christopher Hearse Copyeditor: Jasmine Kwityn Proofreader: Linley Dolby Indexer: Judith McConville Cover Designer: Randy Comer Interior Designer: David Futato Illustrator: Rebecca Demarest September 2013: First Edition Revision History for the First Edition: 2013-09-10: First release See http://oreilly.com/catalog/errata.csp?isbn=9781449358068 for release details. Nutshell Handbook, the Nutshell Handbook logo, and the O’Reilly logo are registered trademarks of O’Reilly Media, Inc. RESTful Web APIs, the image of Hoffmann’s two-toed sloth, and related trade dress are trade‐ marks of O’Reilly Media, Inc. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and O’Reilly Media, Inc., was aware of a trade‐ mark claim, the designations have been printed in caps or initial caps. While every precaution has been taken in the preparation of this book, the publisher and authors assume no responsibility for errors or omissions, or for damages resulting from the use of the information contained herein. ISBN: 978-1-449-35806-8 [LSI]

For Sienna, Dalton, and Maggie. —Leonard For Milo “The Supervisor,” my constant and patient companion throughout this and so many other projects. Thanks, buddy! —Mike

Table of Contents Foreword. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv 1. Surfing the Web. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Episode 1: The Billboard 2 Resources and Representations 2 Addressability 3 Episode 2: The Home Page 3 Short Sessions 4 Self-Descriptive Messages 5 Episode 3: The Link 6 Standardized Methods 8 Episode 4: The Form and the Redirect 9 Application State 10 Resource State 11 Connectedness 13 The Web Is Something Special 14 Web APIs Lag Behind the Web 15 The Semantic Challenge 16 2. A Simple API. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 HTTP GET: Your Safe Bet 18 How to Read an HTTP Response 18 JSON 20 Collection+JSON 21 Writing to an API 22 HTTP POST: How Resources Are Born 24 Liberated by Constraints 25 v

Application Semantics Create the Semantic Gap 27 3. Resources and Representations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 A Resource Can Be Anything 30 A Representation Describes Resource State 30 Representations Are Transferred Back and Forth 31 Resources with Many Representations 32 The Protocol Semantics of HTTP 33 GET 34 DELETE 35 Idempotence 36 POST-to-Append 37 PUT 37 PATCH 38 LINK and UNLINK 39 HEAD 40 OPTIONS 40 Overloaded POST 41 Which Methods Should You Use? 42 4. Hypermedia. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 HTML as a Hypermedia Format 46 URI Templates 49 URI Versus URL 50 The Link Header 51 What Hypermedia Is For 52 Guiding the Request 52 Promises About the Response 53 Workflow Control 54 Beware of Fake Hypermedia! 55 The Semantic Challenge: How Are We Doing? 56 5. Domain-Specific Designs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 Maze+XML: A Domain-Specific Design 60 How Maze+XML Works 61 Link Relations 62 Follow a Link to Change Application State 64 The Collection of Mazes 65 Is Maze+XML an API? 67 Client #1: The Game 68 A Maze+XML Server 72 Client #2: The Mapmaker 74 vi | Table of Contents

Client #3: The Boaster 76 Clients Do the Job They Want to Do 77 Extending a Standard 77 The Mapmaker’s Flaw 80 The Fix (and the Flaw in the Fix) 81 Maze as Metaphor 83 Meeting the Semantic Challenge 83 Where Are the Domain-Specific Designs? 83 The Prize at the End 84 Hypermedia in the Headers 84 Steal the Application Semantics 84 If You Can’t Find a Domain-Specific Design, Don’t Make One 86 Kinds of API Clients 86 Human-Driven Clients 86 Automated Clients 87 6. The Collection Pattern. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 What’s a Collection? 93 Collections Link to Items 93 Collection+JSON 94 Representing the Items 95 The Write Template 98 Search Templates 99 How a (Generic) Collection Works 100 GET 100 POST-to-Append 100 PUT and PATCH 101 DELETE 101 Pagination 101 Search Forms 102 The Atom Publishing Protocol (AtomPub) 102 AtomPub Plug-in Standards 104 Why Doesn’t Everyone Use AtomPub? 105 The Semantic Challenge: How Are We Doing? 106 7. Pure-Hypermedia Designs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 Why HTML? 109 HTML’s Capabilities 110 Hypermedia Controls 110 Plug-in Application Semantics 111 Microformats 113 The hMaze Microformat 114 Table of Contents | vii

Microdata 116 Changing Resource State 117 Adding Application Semantics to Forms 119 The Alternative to Hypermedia Is Media 122 HTML’s Limits 124 HTML 5 to the Rescue? 124 The Hypertext Application Language 125 Siren 129 The Semantic Challenge: How Are We Doing? 130 8. Profiles. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 How Does A Client Find the Documentation? 134 What’s a Profile? 135 Linking to a Profile 135 The profile Link Relation 135 The profile Media Type Parameter 136 Special-Purpose Hypermedia Controls 136 Profiles Describe Protocol Semantics 137 Profiles Describe Application Semantics 138 Link Relations 138 Unsafe Link Relations 139 Semantic Descriptors 140 XMDP: The First Machine-Readable Profile Format 141 ALPS 143 Advantages of ALPS 148 ALPS Doesn’t Do Everything 150 JSON-LD 150 Embedded Documentation 154 In Summary 155 9. The Design Procedure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 Two-Step Design Procedure 157 Seven-Step Design Procedure 158 Step 1: List the Semantic Descriptors 159 Step 2: Draw a State Diagram 161 Step 3: Reconcile Names 164 Step 4: Choose a Media Type 167 Step 5: Write a Profile 169 Step 6: Implementation 169 Step 7: Publication 170 Example: You Type It, We Post It 173 List the Semantic Descriptors 173 viii | Table of Contents

Draw a State Diagram 174 Reconcile Names 174 Choose a Media Type 175 Write a Profile 176 Some Design Advice 177 Resources Are Implementation Details 178 Don’t Fall into the Collection Trap 178 Don’t Start with the Representation Format 179 URL Design Doesn’t Matter 180 Standard Names Are Probably Better Than Your Names 182 If You Design a Media Type 183 When Your API Changes 185 Don’t Keep All the Hypermedia in One Place 189 Adding Hypermedia to an Existing API 190 Fixing Up an XML-Based API 191 Is It Worth It? 192 Alice’s Second Adventure 192 Episode 1: The Nonsense Representation 192 Episode 2: The Profile 194 Alice Figured It Out 196 10. The Hypermedia Zoo. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 Domain-Specific Formats 200 Maze+XML 200 OpenSearch 201 Problem Detail Documents 201 SVG 202 VoiceXML 204 Collection Pattern Formats 206 Collection+JSON 206 The Atom Publishing Protocol 207 OData 208 Pure Hypermedia Formats 215 HTML 215 HAL 216 Siren 217 The Link Header 218 The Location and Content-Location Headers 218 URL Lists 219 JSON Home Documents 219 The Link-Template Header 220 WADL 221 Table of Contents | ix

XLink 222 XForms 223 GeoJSON: A Troubled Type 224 GeoJSON Has No Generic Hypermedia Controls 226 GeoJSON Has No Media Type 228 Learning from GeoJSON 229 The Semantic Zoo 230 The IANA Registry of Link Relations 230 The Microformats Wiki 230 Link Relations from the Microformats Wiki 232 schema.org 233 Dublin Core 234 Activity Streams 234 The ALPS Registry 235 11. HTTP for APIs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237 The New HTTP/1.1 Specification 238 Response Codes 238 Headers 238 Choosing Between Representations 239 Content Negotiation 239 Hypermedia Menus 240 The Canonical URL 241 HTTP Performance 241 Caching 241 Conditional GET 242 Look-Before-You-Leap Requests 244 Compression 245 Partial GET 246 Pipelining 247 Avoiding the Lost Update Problem 248 Authentication 249 The WWW-Authenticate and Authorization Headers 250 Basic Auth 251 OAuth 1.0 252 Where OAuth 1.0 Falls Short 255 OAuth 2.0 256 When to Give Up on OAuth 256 Extensions to HTTP 257 The PATCH Method 257 The LINK and UNLINK Methods 258 WebDAV 259 x | Table of Contents

HTTP 2.0 260 12. Resource Description and Linked Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263 RDF 264 RDF Treats URLs as URIs 265 When to Use the Description Strategy 266 Resource Types 269 RDF Schema 270 The Linked Data Movement 272 JSON-LD 274 JSON-LD as a Representation Format 275 Hydra 276 The XRD Family 280 XRD and JRD 281 Web Host Metadata Documents 282 WebFinger 283 The Ontology Zoo 284 schema.org RDF 284 FOAF 285 vocab.org 285 Conclusion: The Description Strategy Lives! 286 13. CoAP: REST for Embedded Systems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287 A CoAP Request 288 A CoAP Response 288 Kinds of Messages 289 Delayed Response 290 Multicast Messages 291 The CoRE Link Format 291 Conclusion: REST Without HTTP 293 A. The Status Codex. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295 B. The Header Codex. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317 C. An API Designer’s Guide to the Fielding Dissertation. . . . . . . . . . . . . . . . . . . . . . . . . . . . 341 Glossary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357 Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361 Table of Contents | xi

Foreword Progressive Disclosure is a concept in User Interface Design which advocates only pre‐ senting to the user the information they need when they need it. In many ways, the book you are reading right now is an example of this principle. In fact, it is quite likely that this book wouldn’t have “worked” a mere seven years ago. For you see, the programming world was quite a different place when RESTful Web Services, the predecessor of this book, was written. At that time, the term “REST” was was rarely used. And when it was used it was often misapplied, and widely misunder‐ stood. This was the case despite the fact that the standards upon which REST is based, namely HTTP and HTML, were developed and became IETF and W3C standards in roughly their current form in the second half of the 1990s. Roy Fielding’s thesis paper in which he introduced the term REST and on which this book was based was itself published in 2000. Leonard Richardson and I set out to correct this injustice. To do this, we focused pri‐ marily on the concepts underpinning HTTP, and we provided practical guidance on how to apply those concepts to applications. I’d like to think that we helped kick a few pebbles loose that started the avalanche of support for REST that came forth since that time. REST rapidly took on a life of its own, and in the process has become a buzzword. In fact it now is pretty much the case that presenting a web interface and calling it REST is practically the default. We’ve definitely come a long way in a few short years. Admittedly, REST as a term is often over applied, and not always correctly. But all things considered, I am very pleased that the concepts of resources and URIs have successfully managed to infiltrate their way into application interface design. The web, after all, is a resilient place, and these new interfaces, albeit imperfect, are leaps and bounds better than the ones that they replace. But we can do better. xiii

Now that those building blocks are in place, it is time to take a step back, survey the territory, and build on top of these concepts. The next logical step is to explore media types in general, and hypermedia formats in specific. While the first book focused almost exclusively on the correct application of HTTP, it is time to delve more deeply into the concepts behind hypertext media types like HTML—media types that aren’t tightly bound to a single application or even a single vendor. HTML remains a prime example of a such a hypermedia format, and it continues to hold a special place in web architecture. In fact, my personal journey of discovery has been to take a deep dive into development of the W3C standard for HTML, now branded as HTML5. And while HTML does have a prominent place in this new book, there is so much more to cover on the topic of hypermedia. So while I have remained in touch, Leonard picked up a capable replacement for my role as coauthor in Mike Amundsen. It has been a pleasure to watch this book be written, and in reading this book I’ve learned about a number of media types that I had not been exposed to by any other source. More importantly, this book shows what these types have in common, and how to differentiate them, as each has its own specialty. Hopefully the pebbles that this book kicks loose will have the same effect as its prede‐ cessor did. Who knows, perhaps in another seven years it will be time to do this all over again, and highlight some other facet of Representational State Transfer that continues to be under-appreciated. —Sam Ruby xiv | Foreword

Introduction “Most software systems are created with the implicit assumption that the entire system is under the control of one entity, or at least that all entities participating within a system are acting towards a common goal and not at cross-purposes. Such an assumption cannot be safely made when the system runs openly on the Internet.” — Roy Fielding Architectural Styles and the Design of Network-based Software Architectures “A Discordian Shall Always use the Official Discordian Document Numbering System.” — Malaclypse the Younger and Lord Omar Khayyam Ravenhurst Principia Discordia I’m going to show you a better way to do distributed computing, using the ideas un‐ derlying the most successful distributed system in history: the World Wide Web. I hope you’ll read this book if you’ve decided (or your manager has decided) that your company needs to publish a web API. It doesn’t matter whether you’re planning a public API, a purely internal API, or an API accessible by trusted partners—they can all benefit from the philosophy of REST. This is not necessarily the book for you if you want to learn how to write API clients. That’s because most existing API designs are based on assumptions that are several years old, assumptions that I’d like to destroy. Most of today’s APIs have a big problem: once deployed, they can’t change. There are big-name APIs that stay static for years at a time, as the industry changes around them, because changing them would be too difficult. But RESTful architectures are designed for managing change. The World Wide Web is made of millions of websites, running atop thousands of different server implementa‐ tions, and undergoing periodic redesigns. Websites are accessed by billions of users who are using hundreds of different client implementations on dozens of hardware plat‐ xv

forms. Your deployment won’t look like this howling mess, but the closer you come to web scale, the more familiar this picture will look. A very simple system is always easy to change. At small scales, a RESTful system has a larger up-front design cost than a push-button solution. But as your API matures and starts to change, you’ll really need some way—like REST—of adapting to change. • An API that’s commercially successful will stay available for years on end. Some APIs have hundreds or even thousands of users. Even if the problem domain only changes occasionally, the cumulative effect on clients can be huge. • Some APIs change all the time, with new data elements and business rules con‐ stantly being added. • In some APIs, each client can change the workflow to suit its needs. Even if the API itself never changes, each client will experience it differently. • The people who write the API clients usually don’t work on the same team as the people who write the servers. All APIs that are open to the public fall under this category. If you don’t know what kind of clients are out there, you need to be very careful about making changes—or you need to have a design that can change without breaking all the clients. If you copy existing designs for your API, you will probably only repeat the mistakes of the past. Unfortunately, most of the improvements are happening below the surface, in experiments and through slow-moving standards processes. I’ll cover dozens of specific technologies in this book, including many that are still under development. But my main goal is to teach you the underlying principles of REST. Learn those, and you’ll be able to exploit whichever experiments pan out and whichever standards are approved. There are two specific problems I’m trying to solve with this book: duplication of effort and avoidance of hypermedia. Let’s take a look at them. Duplication of Effort An API released today will be named after the company that hosts it. We talk about the “Twitter API,” the “Facebook API,” and the “Google+ API.” These three APIs do similar things. They all have some notion of user accounts and (among other things) they all let users post a little bit of text to their accounts. But each API has a completely different design. Learning one API doesn’t help you learn the next one. Of course, Twitter, Facebook, and Google are big companies that compete with each other. They don’t want to make it easy for you to learn their competitors’ APIs. But small companies and nonprofits do the same thing. They design their APIs as though nobody else had ever had a similar idea. This interferes with their goal of getting people to actually use their APIs. xvi | Introduction