Java 8 Stream API: A Comprehensive Guide for Biomedical Informatics

1. Java 8 Stream API Raj Thavamani Application Developer / Java Group Biomedical Informatics

2. Outline Stream Building Blocks Java 8 Default Methods Functional Interfaces Lambda Expressions Method References

3. Outline Characteristics of Streams Creating Streams Common Functional Interfaces Used Anatomy of the Stream pipeline Optional Class Common Stream API Methods Used Examples Parallel Streams Unbounded (On the Fly) Streams What Could Streams Do For BMI References Questions?

4. Java 8 Target Release Date: 03/18/14 Introduces Default Methods Functional Interfaces Lambda Expressions Stream API and overall improvements to Collections to support Streams

5. Default Methods In Context of Support For Streams Java 8 needed to add functionality to existing Collection interfaces to support Streams (stream(), forEach())

6. Default Methods Problem Pre-Java 8 interfaces couldnt have method bodies. The only way to add functionality to Interfaces was to declare additional methods which would be implemented in classes that implement the interface It is impossible to add methods to an interface without breaking the existing implementation

7. Default Methods Solution Default Methods! Java 8 allows default methods to be added to interfaces with their full implementation Classes which implement the interface dont have to have implementations of the default method Allows the addition of functionality to interfaces while preserving backward compatibility

8. Default Methods Example public interface A { default void foo(){ System.out.println("Calling A.foo()"); } public class Clazz implements A {} Clazz clazz = new Clazz(); clazz.foo(); // Calling A.foo()

9. Functional Interfaces Interfaces with only one abstract method. With only one abstract method, these interfaces can be easily represented with lambda expressions Example @FunctionalInterface public interface SimpleFuncInterface { public void doWork(); }

10. Lambda expressions A more brief and clearly expressive way to implement functional interfaces Format: -> Example (Functional Interface) public interface Predicate { boolean test(T input); } Example (Static Method) public static Collection filter(Predicate predicate, Collection items) { Collection result = new ArrayList(); for(T item: items) { if(predicate.test(item)) { result.add(item); } } } Example (Call with Lambda Expression) Collection myInts = asList(0,1,2,3,4,5,6,7,8,9); Collection onlyOdds = filter( n -> n % 2 != 0 , myInts)

11. Method References Event more brief and clearly expressive way to implement functional interfaces Format: :: Example (Functional Interface) public interface IntPredicates { boolean isOdd(Integer n) { return n % 2 != 0; } } Example (Call with Lambda Expression) List numbers = asList(1,2,3,4,5,6,7,8,9); List odds = filter(n -> IntPredicates.isOdd(n), numbers); Example (Call with Method Reference) List numbers = asList(1,2,3,4,5,6,7,8,9); List odds = filter( IntPredicates::isOdd, numbers);

12. Characteristics of Streams Streams are not related to InputStreams, OutputStreams, etc. Streams are NOT data structures but are wrappers around Collection that carry values from a source through a pipeline of operations. Streams are more powerful, faster and more memory efficient than Lists Streams are designed for lambdas Streams can easily be output as arrays or lists Streams employ lazy evaluation Streams are parallelizable Streams can be on-the-fly

13. Creating Streams From individual values Stream.of(val1, val2, ) From array Stream.of(someArray) Arrays.stream(someArray) From List (and other Collections) someList.stream() someOtherCollection.stream()

14. Common Functional Interfaces Used Predicate Represents a predicate (boolean-valued function) of one argument Functional method is boolean Test(T t) Evaluates this Predicate on the given input argument (T t) Returns true if the input argument matches the predicate, otherwise false Supplier Represents a supplier of results Functional method is T get() Returns a result of type T Function Represents a function that accepts one argument and produces a result Functional method is R apply(T t) Applies this function to the given argument (T t) Returns the function result Consumer Represents an operation that accepts a single input and returns no result Functional method is void accept(T t) Performs this operation on the given argument (T t)

15. Common Functional Interfaces Used Function Represents an operation that accepts one argument and produces a result Functional method is R apply(T t) Applies this function to the given argument (T t) Returns the function result UnaryOperator Represents an operation on a single operands that produces a result of the same type as its operand Functional method is R Function.apply(T t) Applies this function to the given argument (T t) Returns the function result

16. Common Functional Interfaces Used BiFunction Represents an operation that accepts two arguments and produces a result Functional method is R apply(T t, U u) Applies this function to the given arguments (T t, U u) Returns the function result BinaryOperator Extends BiFunction Represents an operation upon two operands of the same type, producing a result of the same type as the operands Functional method is R BiFunction.apply(T t, U u) Applies this function to the given arguments (T t, U u) where R,T and U are of the same type Returns the function result Comparator Compares its two arguments for order. Functional method is int compareTo(T o1, T o2) Returns a negative integer, zero, or a positive integer as the first argument is less than, equal to, or greater than the second.

17. Anatomy of the Stream Pipeline A Stream is processed through a pipeline of operations A Stream starts with a source data structure Intermediate methods are performed on the Stream elements. These methods produce Streams and are not processed until the terminal method is called. The Stream is considered consumed when a terminal operation is invoked. No other operation can be performed on the Stream elements afterwards A Stream pipeline contains some short-circuit methods (which could be intermediate or terminal methods) that cause the earlier intermediate methods to be processed only until the short-circuit method can be evaluated .

18. Anatomy of the Stream Pipeline Intermediate Methods map, filter, distinct, sorted, peek, limit, parallel Terminal Methods forEach, toArray, reduce, collect, min, max, count, anyMatch, allMatch, noneMatch, findFirst, findAny, iterator Short-circuit Methods anyMatch, allMatch, noneMatch, findFirst, findAny,limit

19. Optional Class A container which may or may not contain a non-null value Common methods isPresent() returns true if value is present Get() returns value if present orElse(T other) returns value if present, or other ifPresent(Consumer) runs the lambda if value is present

20. Common Stream API Methods Used Void forEach(Consumer) Easy way to loop over Stream elements You supply a lambda for forEach and that lambda is called on each element of the Stream Related peek method does the exact same thing, but returns the original Stream

21. Common Stream API Methods Used Void forEach(Consumer) Example Employees.forEach(e -> e.setSalary(e.getSalary() * 11/10)) Give all employees a 10% raise

22. Common Stream API Methods Used Void forEach(Consumer) Vs. For Loops List employees = getEmployees(); for(Employee e: employees) { e.setSalary(e.getSalary() * 11/10); } Advantages of forEach Designed for lambdas to be marginally more succinct Lambdas are reusable Can be made parallel with minimal effort

23. Common Stream API Methods Used Stream map(Function) Produces a new Stream that is the result of applying a Function to each element of original Stream Example Ids.map(EmployeeUtils::findEmployeeById) Create a new Stream of Employee ids

24. Common Stream API Methods Used Stream filter(Predicate) Produces a new Stream that contains only the elements of the original Stream that pass a given test Example employees.filter(e -> e.getSalary() > 100000) Produce a Stream of Employees with a high salary

25. Common Stream API Methods Used Optional findFirst() Returns an Optional for the first entry in the Stream Example employees.filter().findFirst().orElse(Consult ant) Get the first Employee entry that passes the filter

26. Common Stream API Methods Used Object[] toArray(Supplier) Reads the Stream of elements into a an array Example Employee[] empArray = employees.toArray(Employee[]::new); Create an array of Employees out of the Stream of Employees

27. Common Stream API Methods Used List collect(Collectors.toList()) Reads the Stream of elements into a List or any other collection Example List empList = employees.collect(Collectors.toList()); Create a List of Employees out of the Stream of Employees

28. Common Stream API Methods Used List collect(Collectors.toList()) partitioningBy You provide a Predicate. It builds a Map where true maps to a List of entries that passed the Predicate, and false maps to a List that failed the Predicate. Example Map> richTable = googlers().collect (partitioningBy(e -> e.getSalary() > 1000000)); groupingBy You provide a Function. It builds a Map where each output value of the Function maps to a List of entries that gave that value. Example Map> deptTable = employeeStream().collect(groupingBy(Employee::getDepartment));

29. Common Stream API Methods Used T reduce(T identity, BinaryOperator) You start with a seed (identity) value, then combine this value with the first Entry in the Stream, combine the second entry of the Stream, etc. Example Nums.stream().reduce(1, (n1,n2) -> n1*n2) Calculate the product of numbers IntStream (Stream on primative int] has build-in sum() Built-in Min, Max methods

30. Common Stream API Methods Used Stream limit(long maxSize) Limit(n) returns a stream of the first n elements Example someLongStream.limit(10) First 10 elements

31. Common Stream API Methods Used Stream skip(long n) skip(n) returns a stream starting with element n Example twentyElementStream.skip(5) Last 15 elements

32. Common Stream API Methods Used Stream sorted(Comparator) Returns a stream consisting of the elements of this stream, sorted according to the provided Comparator Example empStream.map().filter().limit() .sorted((e1, e2) -> e1.getSalary() - e2.getSalary()) Employees sorted by salary

33. Common Stream API Methods Used Optional min(Comparator) Returns the minimum element in this Stream according to the Comparator Example Employee alphabeticallyFirst = ids.stream().map(EmployeeSamples::findGoogler) .min((e1, e2) -> e1.getLastName() .compareTo(e2.getLastName())) .get(); Get Googler with earliest lastName

34. Common Stream API Methods Used Optional max(Comparator) Returns the minimum element in this Stream according to the Comparator Example Employee richest = ids.stream().map(EmployeeSamples::findGoogler) .max((e1, e2) -> e1.getSalary() - e2.getSalary()) .get(); Get Richest Employee

35. Common Stream API Methods Used Stream distinct() Returns a stream consisting of the distinct elements of this stream Example List ids2 = Arrays.asList(9, 10, 9, 10, 9, 10); List emps4 = ids2.stream().map(EmployeeSamples::findGoogler) .distinct() .collect(toList()); Get a list of distinct Employees

36. Common Stream API Methods Used Boolean anyMatch(Predicate), allMatch(Predicate), noneMatch(Predicate) Returns true if Stream passes, false otherwise Lazy Evaluation anyMatch processes elements in the Stream one element at a time until it finds a match according to the Predicate and returns true if it found a match allMatch processes elements in the Stream one element at a time until it fails a match according to the Predicate and returns false if an element failed the Predicate noneMatch processes elements in the Stream one element at a time until it finds a match according to the Predicate and returns false if an element matches the Predicate Example employeeStream.anyMatch(e -> e.getSalary() > 500000) Is there a rich Employee among all Employees?

37. Common Stream API Methods Used long count() Returns the count of elements in the Stream Example employeeStream.filter(somePredicate).c ount() How many Employees match the criteria?

38. Parallel Streams Helper Methods For Timing private static void timingTest(Stream testStream) { long startTime = System.nanoTime(); testStream.forEach(e -> doSlowOp()); long endTime = System.nanoTime(); System.out.printf(" %.3f seconds.%n", deltaSeconds(startTime, endTime)); } private static double deltaSeconds(long startTime, long endTime) { return((endTime - startTime) / 1000000000); }

39. Parallel Streams Helper Method For Simulating Long Operation void doSlowOp() { try { TimeUnit.SECONDS.sleep(1); } catch (InterruptedException ie) { // Nothing to do here. } }

40. Parallel Streams Main Code System.out.print("Serial version [11 entries]:"); timingTest(googlers()); int numProcessorsOrCores = Runtime.getRuntime().availableProcessors(); System.out.printf("Parallel version on %s-core machine:", numProcessorsOrCores); timingTest(googlers(). parallel() );

41. Parallel Streams Results Serial version [11 entries]: 11.000 seconds. Parallel version on 4-core machine: 3.000 seconds.

42. (On The Fly) Streams Stream generate(Supplier) The method lets you specify a Supplier This Supplier is invoked each time the system needs a Stream element Example List emps = Stream.generate(() -> randomEmployee()) .limit(n) .collect(toList()); Stream iterate(T seed, UnaryOperator f) The method lets you specify a seed and a UnaryOperator. The seed becomes the first element of the Stream, f(seed) becomes the second element of the Stream, f(second) becomes the third element, etc. Example List powersOfTwo = Stream.iterate(1, n -> n * 2) .limit(n) .collect(toList()); The values are not calculated until they are needed To avoid unterminated processing, you must eventually use a size-limiting method This is less of an actual Unbounded Stream and more of an On The Fly Stream

43. What Could Streams do For BMI? The real excitement with Streams is when you combine Stream operators into one pipeline Parallel Processing on large Patient Sets Taking advantage of groupingBy and partitioningBy to perform analysis Example1: PvpPatientPicker for ICN A massive datatable that needs to have the ability to filter on any column as well as do nested filtering Think of how much code you would need to implement the filtering Using Streams: List pvpPatients = . List filteredPvpPatients = Stream.of(PvpPatients) .parallel() .map(PvpPatient::findByPatientNumber) .filter(pvpPatient -> pvpPatient.ProviderId == 101) .filter(pvpPatient -> pvpPatient.careStratificationScore == 10) .collect(Collectors.toList()); Example2: QueryTool for i2b2 Using Streams: Map> patientsByRace = patientStream().collect(groupingBy(Patient::getRace));

44. References Stream API http://download.java.net/jdk8/docs/api/java/util/stream/S tream.html Java 8 Explained: Applying Lambdas to Java Collections http://zeroturnaround.com/rebellabs/java-8-explained- applying-lambdas-to-java-collections/ Java 8 first steps with Lambdas and Streams https://blog.codecentric.de/en/2013/10/java-8-first-steps- lambdas-streams/ Java 8Tutorial: Lambda Expressions, Streams, and More http://www.coreservlets.com/java-8-tutorial/

45. Questions?