Java 8 in Action Lambdas, Streams, and functional-style programming (by Raoul-Gabriel Urma, Mario Fusco etc.) (Z-Library)

2 Java 8 in Action: Lambdas, streams, and functional-style programming Raoul-Gabriel Urma, Mario Fusco, and Alan Mycroft

3 Copyright For online information and ordering of this and other Manning books, please visit www.manning.com. The publisher offers discounts on this book when ordered in quantity. For more information, please contact Special Sales Department Manning Publications Co. 20 Baldwin Road PO Box 761 Shelter Island, NY 11964 Email: orders@manning.com ©2015 by Manning Publications Co. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by means electronic, mechanical, photocopying, or otherwise, without prior written permission of the publisher. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in the book, and Manning Publications was aware of a trademark claim, the designations have been printed in initial caps or all caps. Recognizing the importance of preserving what has been written, it is Manning’s policy to have the books we publish printed on acid-free paper, and we exert our best efforts to that end. Recognizing also our responsibility to conserve the resources of our planet, Manning books are printed on paper that is at least 15 percent recycled and processed without the use of elemental chlorine. Manning Publications Co. 20 Baldwin Road PO Box 761 Shelter Island, NY 11964 Development editor: Susan Conant Technical development editor Al Scherer Copyeditor: Linda Recktenwald Proofreader: Katie Tennant Typesetter: Dennis Dalinnik Cover designer: Maria Tudor

4 ISBN: 9781617291999 Printed in the United States of America 1 2 3 4 5 6 7 8 9 10 – EBM – 19 18 17 16 15 14

5 Dedication To our parents

6 Table of Contents Copyright.............................................................................................................................................. 3 Dedication.............................................................................................................................................5 Part 1. Fundamentals...................................................................................................................11 Chapter 1. Java 8: why should you care?............................................................................ 12 1.1. Why is Java still changing?................................................................................................. 14 1.2. Functions in Java.................................................................................................................21 1.3. Streams................................................................................................................................. 28 1.4. Default methods.................................................................................................................. 32 1.5. Other good ideas from functional programming............................................................34 1.6. Summary...............................................................................................................................36 Chapter 2. Passing code with behavior parameterization.......................................... 37 2.1. Coping with changing requirements.................................................................................38 2.2. Behavior parameterization.................................................................................................41 2.3. Tackling verbosity............................................................................................................... 47 2.4. Real-world examples.......................................................................................................... 52 2.5. Summary.............................................................................................................................. 54 Chapter 3. Lambda expressions.............................................................................................56 3.1. Lambdas in a nutshell......................................................................................................... 57 3.2. Where and how to use lambdas........................................................................................60 3.3. Putting lambdas into practice: the execute around pattern......................................... 66 3.4. Using functional interfaces................................................................................................70 3.5. Type checking, type inference, and restrictions..............................................................76 3.6. Method references..............................................................................................................82 3.7. Putting lambdas and method references into practice!.................................................89 3.8. Useful methods to compose lambda expressions...........................................................91 3.9. Similar ideas frommathematics.......................................................................................95 3.10. Summary............................................................................................................................98 Part 2. Functional-style data processing............................................................................99

7 Chapter 4. Introducing streams.......................................................................................... 100 4.1. What are streams?.............................................................................................................100 4.2. Getting started with streams........................................................................................... 105 4.3. Streams vs. collections..................................................................................................... 108 4.4. Stream operations..............................................................................................................113 4.5. Summary............................................................................................................................. 117 Chapter 5. Working with streams........................................................................................118 5.1. Filtering and slicing........................................................................................................... 119 5.2. Mapping..............................................................................................................................123 5.3. Finding and matching.......................................................................................................129 5.4. Reducing............................................................................................................................. 132 5.5. Putting it all into practice.................................................................................................140 5.6. Numeric streams............................................................................................................... 145 5.7. Building streams................................................................................................................ 152 5.8. Summary............................................................................................................................ 158 Chapter 6. Collecting data with streams...........................................................................159 6.1. Collectors in a nutshell..................................................................................................... 160 6.2. Reducing and summarizing.............................................................................................163 6.3. Grouping............................................................................................................................. 172 6.4. Partitioning........................................................................................................................180 6.5. The Collector interface..................................................................................................... 186 6.6. Developing your own collector for better performance...............................................194 6.7. Summary............................................................................................................................202 Chapter 7. Parallel data processing and performance...............................................203 7.1. Parallel streams................................................................................................................. 204 7.2. The fork/join framework..................................................................................................214 7.3. Spliterator.......................................................................................................................... 222 7.4. Summary............................................................................................................................232 Part 3. Effective Java 8 programming.............................................................................. 233 Chapter 8. Refactoring, testing, and debugging...........................................................234 8.1. Refactoring for improved readability and flexibility....................................................234

8 8.2. Refactoring object-oriented design patterns with lambdas....................................... 242 8.3. Testing lambdas................................................................................................................253 8.4. Debugging..........................................................................................................................256 8.5. Summary............................................................................................................................ 261 Chapter 9. Default methods..................................................................................................262 9.1. Evolving APIs.....................................................................................................................265 9.2. Default methods in a nutshell.........................................................................................269 9.3. Usage patterns for default methods............................................................................... 272 9.4. Resolution rules.................................................................................................................277 9.5. Summary............................................................................................................................284 Chapter 10. Using Optional as a better alternative to null.......................................285 10.1. How do you model the absence of a value?.................................................................286 10.2. Introducing the Optional class..................................................................................... 290 10.3. Patterns for adopting Optional.....................................................................................292 10.4. Practical examples of using Optional...........................................................................303 10.5. Summary..........................................................................................................................307 Chapter 11. CompletableFuture: composable asynchronous programming....309 11.1. Futures................................................................................................................................311 11.2. Implementing an asynchronous API.............................................................................314 11.3. Make your code non-blocking....................................................................................... 320 11.4. Pipelining asynchronous tasks...................................................................................... 328 11.5. Reacting to a CompletableFuture completion.............................................................338 11.6. Summary...........................................................................................................................342 Chapter 12. New Date and Time API..................................................................................343 12.1. LocalDate, LocalTime, Instant, Duration, and Period...............................................344 12.2. Manipulating, parsing, and formatting dates.............................................................350 12.3. Working with different time zones and calendars..................................................... 358 12.4. Summary...........................................................................................................................361 Part 4. Beyond Java 8.............................................................................................................. 363 Chapter 13. Thinking functionally......................................................................................364 13.1. Implementing and maintaining systems......................................................................364

9 13.2. What’s functional programming?.................................................................................368 13.3. Recursion vs. iteration....................................................................................................375 13.4. Summary.......................................................................................................................... 379 Chapter 14. Functional programming techniques....................................................... 381 14.1. Functions everywhere..................................................................................................... 381 14.2. Persistent data structures.............................................................................................. 385 14.3. Lazy evaluation with streams........................................................................................392 14.4. Pattern matching.............................................................................................................401 14.5. Miscellany........................................................................................................................ 407 14.6. Summary.......................................................................................................................... 410 Chapter 15. Blending OOP and FP: comparing Java 8 and Scala...........................412 15.1. Introduction to Scala....................................................................................................... 413 15.2. Functions..........................................................................................................................422 15.3. Classes and traits.............................................................................................................427 15.4. Summary.......................................................................................................................... 429 Chapter 16. Conclusions and where next for Java....................................................... 431 16.1. Review of Java 8 features................................................................................................431 16.2. What’s ahead for Java?...................................................................................................435 16.3. The final word..................................................................................................................446 Appendix A. Miscellaneous language updates...............................................................447 A.1. Annotations........................................................................................................................447 A.2. Generalized target-type inference..................................................................................450 Appendix B. Miscellaneous library updates...................................................................452 B.1. Collections..........................................................................................................................452 B.2. Concurrency...................................................................................................................... 455 B.3. Arrays.................................................................................................................................458 B.4. Number and Math............................................................................................................459 B.5. Files.................................................................................................................................... 460 B.6. Reflection.......................................................................................................................... 460 B.7. String..................................................................................................................................460 Appendix C. Performing multiple operations in parallel on a stream................462

10 C.1. Forking a stream............................................................................................................... 462 C.2. Performance considerations........................................................................................... 472 Appendix D. Lambdas and JVM bytecode.......................................................................473 D.1. Anonymous classes...........................................................................................................473 D.2. Bytecode generation.........................................................................................................473 D.3. InvokeDynamic to the rescue......................................................................................... 475 D.4. Code-generation strategies............................................................................................. 477 Index...............................................................................................................................................479

11 Part 1. Fundamentals This first part of the book provides the fundamentals to help you get started with Java 8. By the end of this first part, you’ll have a full understanding of what lambda expressions are, and you’ll be able to write code that’s both concise and flexible enough to easily adapt to changing requirements. In chapter 1, we summarize the main changes to Java (lambda expressions, method references, streams, and default methods) and set the scene for the book. In chapter 2, you’ll learn about behavior parameterization, a software development pattern that Java 8 relies heavily on and is the motivation for lambda expressions. Chapter 3 gives a full explanation, with code examples and quizzes at every step, of the concepts of lambda expressions and method references.

12 Chapter 1. Java 8: why should you care? This chapter covers  Why Java is changing again  Changing computing background: multicore and processing large datasets (big data)  Pressure to evolve: new architectures favor functional style over imperative  Introducing core new features of Java 8: lambdas, streams, default methods Since the release of JDK 1.0 (Java 1.0) in 1996, Java has won a large following of students, project managers, and programmers who are active users. It’s an expressive language and continues to be used for projects both large and small. Its evolution (via the addition of new features) from Java 1.1 (1997) to Java 7 (2011) has been well managed. Java 8 was released in March 2014. So the question is this: why should you care about Java 8? We argue that the changes to Java 8 are in many ways more profound than any other changes to Java in its history. The good news is that the changes enable you to write programs more easily—instead of writing verbose code like the following (to sort a list of apples in inventory based on their weight), Collections.sort(inventory, new Comparator<Apple>() { public int compare(Apple a1, Apple a2){ return a1.getWeight().compareTo(a2.getWeight()); } }); in Java 8 you can write more concise code that reads a lot closer to the problem statement: It reads “sort inventory comparing apple weight.” Don’t worry about this code for now. This book will explain what it does and how you can write similar code! There’s also a hardware influence: commodity CPUs have become multicore—the processor in your laptop or desktop machine probably has four or more CPU cores within it. But the vast majority of existing Java programs use only one of these cores and leave the other three idle (or spend a small fraction of their processing power running part of the operating system or a virus checker).

13 Prior to Java 8, experts might tell you that you have to use threads to use these cores. The problem is that working with threads is difficult and error prone. Java has followed an evolutionary path of continually trying to make concurrency easier and less error prone. Java 1.0 had threads and locks and even a memory model—the best practice at the time—but these primitives proved too difficult to use reliably in nonspecialist project teams. Java 5 added industrial-strength building blocks like thread pools and concurrent collections. Java 7 added the fork/join framework, making parallelism more practical but still difficult. Java 8 has a new, simpler way of thinking about parallelism. But you still have to follow some rules, which you’ll learn in this book! From these two examples (more concise code and simpler use of multicore processors) springs the whole consistent edifice that is Java 8. We start by giving you a quick taste of these ideas (hopefully enough to intrigue you, but short enough to summarize them):  The Streams API  Techniques for passing code to methods  Default methods in interfaces Java 8 provides a new API (called Streams) that supports many parallel operations to process data and resembles the way you might think in database query languages—you express what you want in a higher-level manner, and the implementation (here the Streams library) chooses the best low-level execution mechanism. As a result, it avoids the need for you to write code that uses synchronized, which is not only highly error prone but is also more expensive than you may realize on multicore CPUs.[1] 1 Multicore CPUs have separate caches (fast memory) attached to each processor core. Locking requires these to be synchronized, requiring relatively slow cache-coherency-protocol intercore communication. From a slightly revisionist viewpoint, the addition of Streams in Java 8 can be seen as a direct cause of the two other additions to Java 8: concise techniques to pass code to methods (method references, lambdas) and default methods in interfaces. But thinking of passing code to methods as a mere consequence of Streams downplays its range of uses within Java 8. It gives you a new concise way to express behavior parameterization. Suppose you want to write two methods that differ in only a few lines of code; you can now just pass the code of the parts that differ as an argument (this programming technique is shorter, clearer, and less error prone than the common tendency to use copy and paste). Experts will here note that behavior parameterization could, prior to Java 8, be encoded using anonymous

14 classes—but we’ll let the example on the first page of this chapter, which shows increased code conciseness with Java 8, speak for itself in terms of clarity! The Java 8 feature of passing code to methods (and also being able to return it and incorporate it into data structures) also provides access to a whole range of additional techniques that are commonly referred to as functional-style programming. In a nutshell, such code, called functions in the functional programming community, can be passed around and combined in a way to produce powerful programming idioms that you’ll see in Java 8 guise throughout this book. The meat of this chapter begins with a high-level discussion on why languages evolve, continues with sections on the core features of Java 8, and then introduces the ideas of functional-style programming that the new features simplify using and that new computer architectures favor. In essence, section 1.1 discusses the evolution process and the concepts, which Java was previously lacking, to exploit multicore parallelism in an easy way. Section 1.2 explains why passing code to methods in Java 8 is such a powerful new programming idiom, and section 1.3 does the same for Streams—the new Java 8 way of representing sequenced data and flexibly indicating whether these can be processed in parallel. Section 1.4 explains how the new Java 8 feature of default methods enables interfaces and their libraries to evolve with less fuss and less recompilation. Finally, section 1.5 looks ahead at the ideas of functional-style programming in Java and other languages sharing the JVM. In summary, this chapter introduces ideas that are successively elaborated in the rest of the book. Enjoy the ride! 1.1. Why is Java still changing? With the 1960s came the quest for the perfect programming language. Peter Landin, famous computer scientist of his day, noted in 1966 in a landmark article[2] that there had already been 700 programming languages and speculated on what the next 700 would be like—including arguments for functional-style programming similar to that in Java 8. 2 P. J. Landin, “The Next 700 Programming Languages,” CACM 9(3):157–65, March 1966. Many thousands of programming languages later, academics have concluded that programming languages behave like an ecosystem: new languages appear and old languages are supplanted unless they evolve. We all hope for a perfect universal language, but in reality certain languages are better fitted for certain niches. For example, C and C++ remain popular for building operating systems and various other embedded systems because of their small run-time footprint and in spite of their lack of programming safety. This lack of safety can lead to programs crashing unpredictably and exposing security holes for viruses and the like; indeed,

15 type-safe languages such as Java and C# have supplanted C and C++ in various applications when the additional run-time footprint is acceptable. Prior occupancy of a niche tends to discourage competitors. Changing to a new language and tool chain is often too painful for just a single feature, but newcomers will eventually displace existing languages, unless they evolve fast enough to keep up (older readers are often able to quote a range of such languages in which they’ve previously coded but whose popularity has since waned—Ada, Algol, COBOL, Pascal, Delphi, and SNOBOL, to name but a few). You’re a Java programmer, and Java has been successful at colonizing (and displacing competitor languages in) a large ecosystem niche of programming tasks for the last 15 years. Let’s examine some reasons for that. 1.1.1. Java’s place in the programming language ecosystem Java started well. Right from the start, it was a well-designed object-oriented language with many useful libraries. It also supported small-scale concurrency from day one, with its integrated support for threads and locks (and with its early prescient acknowledgement, in the form of a hardware-neutral memory model, that concurrent threads on multicore processors can have unexpected behaviors in addition to those that happen on single-core processors). Also, the decision to compile Java to JVM bytecode (a virtual machine code that soon every browser supported) meant that it became the language of choice for internet applet programs (do you remember applets?). Indeed, there’s a danger that the Java virtual machine (JVM) and its bytecode will be seen as more important than the Java language itself and that, for certain applications, Java might be replaced by one of its competing languages such as Scala or Groovy, which also run on the JVM. Various recent updates to the JVM (for example, the new invokedynamic bytecode in JDK7) aim to help such competitor languages run smoothly on the JVM—and to interoperate with Java. Java has also been successful at colonizing various aspects of embedded computing (everything from smartcards, toasters, and settop boxes to car braking systems). How did Java get into a general programming niche? Object orientation became fashionable in the 1990s for two reasons: its encapsulation discipline resulted in fewer software engineering issues than those of C; and as a mental model it easily captured the WIMP programming model of Windows 95 and up. This can be summarized as follows: everything is an object; and a mouse click sends an event message to a handler (invokes the Clicked method in a Mouse object). The write-once run-anywhere model of Java and the

16 ability of early browsers to (safely) execute Java code applets gave it a niche in universities, whose graduates then populated industry. There was initial resistance to the additional run cost of Java over C/C++, but machines got faster and programmer time became more and more important. Microsoft’s C# further validated the Java-style object-oriented model. But the climate is changing for the programming language ecosystem; programmers are increasingly dealing with so-called big data (datasets of terabytes and up) and wishing to exploit multicore computers or computing clusters effectively to process it. And this means using parallel processing—something Java wasn’t previously friendly to. You may have come across programming ideas from other programming niches (for example, Google’s map-reduce or the relative ease of data manipulation using database query languages such as SQL) that help you work with large volumes of data and multicore CPUs. Figure 1.1 summarizes the language ecosystem pictorially: think of the landscape as the space of programming problems and the dominant vegetation for a particular bit of ground as the favorite language for that program. Climate change is the idea that new hardware or new programming influences (for example, “Why can’t I program in SQL-like style?”) mean that different languages become the language of choice for new projects, just like increasing regional temperatures mean grapes now thrive in higher latitudes. But of course there’s hysteresis—many an old farmer will keep raising traditional crops. In summary, new languages are appearing and becoming increasingly popular because they’ve adapted quickly to the climate change.

17 Figure 1.1. Programming languages ecosystem and climate change The main benefit of Java 8 to a programmer is that it provides more programming tools and concepts to solve new or existing programming problems more quickly or, more importantly, in a more concise, more easily maintainable way. Although the concepts are new to Java, they’ve proved powerful in niche research-like languages. We highlight and develop the ideas behind three such programming concepts that have driven the development of the Java 8 features to exploit parallelism and write more concise code in general. We introduce them in a slightly different order from the rest of the book to enable a Unix-based analogy and to expose the “need this because of that” dependencies in Java 8’s new parallelism for multicore. 1.1.2. Stream processing The first programming concept is stream processing. For introductory purposes, a stream is a sequence of data items that are conceptually produced one at a time—a program might read items from an input stream one by one and similarly write items to an output stream. The output stream of one program could well be the input stream of another. One practical example is in Unix or Linux, where many programs operate by reading data from standard input (stdin in Unix and C, System.in in Java), operating on it, and then writing their results to standard output (stdout in Unix and C, System.out in Java). First, a little background: Unix cat creates a stream by concatenating two files, tr translates the characters in a stream, sort sorts lines in a stream, and tail -3 gives the last three lines in a stream. The Unix command line allows such programs to be linked together with pipes (|), giving examples such as

18 cat file1 file2 | tr "[A-Z]" "[a-z]" | sort | tail -3 which (supposing file1 and file2 contain a single word per line) prints the three words from the files that appear latest in dictionary order, after first translating them to lowercase. We say that sort takes a stream of lines[3] as input and produces another stream of lines as output (the latter being sorted), as illustrated in figure 1.2. Note that in Unix the commands (cat, tr, sort, and tail) are executed concurrently, so that sort can be processing the first few lines before cat or tr has finished. A more mechanical analogy is a car-manufacturing assembly line where a stream of cars is queued between processing stations that each take a car, modify it, and pass it on to the next station for further processing; processing at separate stations is typically concurrent even though the assembly line is physically a sequence. 3 Purists will say a “stream of characters,” but it’s conceptually simpler to think that sort reorders lines. Figure 1.2. Unix commands operating on streams Java 8 adds a Streams API (note the uppercase S) in java.util.stream based on this idea; Stream<T> is a sequence of items of type T. You can think of it as a fancy iterator for now. The Streams API has many methods that can be chained to form a complex pipeline just like Unix commands were chained in the previous example. The key motivation for this is that you can now program in Java 8 at a higher level of abstraction, structuring your thoughts of turning a stream of this into a stream of that (similarly to how you think when writing database queries) rather than one item at a time. Another advantage is that Java 8 can transparently run your pipeline of Stream operations on several CPU cores on disjoint parts of the input—this is parallelism almost for free instead of hard work using Threads. We cover the Java 8 Streams API in detail in chapters 4–7. 1.1.3. Passing code to methods with behavior parameterization The second programming concept added to Java 8 is the ability to pass a piece of code to an API. This sounds awfully abstract. In the Unix example, you might want to tell the sort command to

19 use a custom ordering. Although the sort command supports command-line parameters to perform various predefined kinds of sorting such as reverse order, these are limited. For example, let’s say you have a collection of invoice IDs with format similar to 2013UK0001, 2014US0002, .... The first four digits represent the year, the next two letters a country code, and last four digits the ID of a client. You may want to sort these invoice IDs by year or perhaps using the customer ID or even the country code. What you really want is the ability to tell the sort command to take as an argument an ordering defined by the user: a separate piece of code passed to the sort command. Now, as a direct parallel in Java, you want to tell a sort method to compare using a customized order. You could write a method compareUsingCustomerId to compare two invoice IDs but, prior to Java 8, you couldn’t pass this method to another method! You could create a Comparator object to pass to the sort method as we showed at the start of this chapter, but this is verbose and obfuscates the idea of simply reusing an existing piece of behavior. Java 8 adds the ability to pass methods (your code) as arguments to other methods. Figure 1.3, based on figure 1.2, illustrates this idea. We also refer to this conceptually as behavior parameterization. Why is this important? The Streams API is built on the idea of passing code to parameterize the behavior of its operations, just as you passed compareUsingCustomerId to parameterize the behavior of sort. Figure 1.3. Passing method compareUsingCustomerId as an argument to sort We summarize how this works in section 1.2 of this chapter but leave full details to chapters 2 and 3. Chapters 13 and 14 look at more advanced things you can do using this feature, with techniques from the functional programming community.

20 1.1.4. Parallelism and shared mutable data The third programming concept is rather more implicit and arises from the phrase “parallelism almost for free” in our previous discussion on stream processing. What do you have to give up? You may have to make some small changes in the way you code the behavior passed to stream methods. At first, these changes might feel a little uncomfortable, but once you get used to them, you’ll love them. You must provide behavior that is safe to execute concurrently on different pieces of the input. Typically this means writing code that doesn’t access shared mutable data to do its job. Sometimes these are referred to as pure functions or side-effect-free functions or stateless functions, and we’ll discuss these in detail in chapters 7 and 13. The previous parallelism arises only by assuming that multiple copies of your piece of code can work independently. If there’s a shared variable or object, which is written to, then things no longer work: what if two processes want to modify the shared variable at the same time? (Section 1.3 gives a more detailed explanation with a diagram.) You’ll find more about this style throughout the book. Java 8 streams exploit parallelism more easily than Java’s existing Threads API, so although it’s possible to use synchronized to break the no-shared-mutable-data rule, it’s fighting the system in that it’s abusing an abstraction optimized around that rule. Using synchronized across multiple processing cores is often far more expensive than you expect, because synchronization forces code to execute sequentially, which works against the goal of parallelism. Two of these points (no shared mutable data and the ability to pass methods and functions—code—to other methods) are the cornerstones of what’s generally described as the paradigm of functional programming, which you’ll see in detail in chapters 13 and 14. In contrast, in the imperative programming paradigm you typically describe a program in terms of a sequence of statements that mutate state. The no-shared-mutable-data requirement means that a method is perfectly described solely by the way it transforms arguments to results; in other words, it behaves as a mathematical function and has no (visible) side effects. 1.1.5. Java needs to evolve You’ve seen evolution in Java before. For example, the introduction of generics and using List<String> instead of just List may initially have been irritating. But you’re now familiar with this style and the benefits it brings (catching more errors at compile time and making code easier to read, because you now know what something is a list of). Other changes have made common things easier to express, for example, using a for-each loop instead of exposing the boilerplate use of an Iterator. The main changes in Java 8 reflect a move