Estimate project
← All posts

Stream Gatherers in Practice, Part 1

The first step in our journey through Stream Gatherers, a compelling new feature of Java 22, with a focus on building custom Gatherers that solve real-world problems.

10 min read
JavaFunctional Programminguncoverjavajava22stream gatherersstream

Java 22 brings a compelling preview feature, JEP 461: Stream Gatherers, which is a key upgrade to the Stream API. At its heart is a new gather method that makes the API more customizable, flexible, and efficient. Its goals are to:

  • Make the Stream API more adaptable, enabling stream operations that better fit specific needs.
  • Improve the reusability of code for stream-based tasks.
  • Make complex stream operations easier to understand and implement.

Importantly, all of this is achieved without making the existing Stream API much more complicated.

This exploration comes in two parts: this article covers the fundamental concepts of Gatherers and how to apply them in practical operations. The next one will look at built-in gatherers and how to combine them into complex operations.

Understanding the Gatherer Interface

The Gatherer interface lies at the core of JEP 461 and makes it possible to customize stream operations in Java 22. It is made up of a few key methods that support complex data processing in a streamlined, efficient way:

  • Initializer Method: Optional, and especially relevant when the integrator or finisher uses an internal state. In that case, the Initializer Method is in charge of correctly initializing this internal state before any data is processed.
  • Integrator Method: Always has to be implemented. This is the central method that defines the actual data processing logic. Every Gatherer needs it to specify how each stream element is processed and folded into the current state.
  • Finisher Method: Implementing it is optional. It handles the final processing and transformations once all elements have been integrated.
  • Combiner Method: Overriding it is optional. This method is key for parallel stream operations, as it specifies how results from different stream segments are merged. It matters when the stream is processed in parallel and should be adapted to your particular parallel processing needs.
  • AndThen Method: Overriding it is optional. The default implementation lets you build a gatherer out of other gatherers.

The Stream::gather method works hand in hand with the Gatherer interface, enabling a more expressive and flexible way of processing streams. By handing a Gatherer instance to Stream::gather, developers can build custom operations suited to their particular needs, going beyond the limits of the Stream API’s predefined operations.

Implementation of custom Gatherers

In this section, we’ll look at how the gather method added to the Stream API in Java 22 can be used in practice. To show how versatile and powerful it is, we’ll build several custom gatherers working on a purpose-built Money class. These examples will show how the gather method can simplify and improve operations on complex data types, shedding light on both the practical value and the innovative potential of this new feature. As noted earlier, Stream Gatherers are still in preview, so the functionality has to be enabled with the JVM flag –enable-preview.

public record Money(BigDecimal amount, Currency currency) {
   public Money add(Money money) {
       return new Money(money.amount.add(this.amount), currency);
   }

   public Money multiply(BigDecimal multiplier) {
       return new Money(amount.multiply(multiplier), currency);
   }
}

Distinct by Gatherer

To show what the gather method is good for, let’s begin with a DistinctByGatherer. It keeps stream elements unique according to a selector function – something the Stream API doesn’t support out of the box.

public class DistinctByGatherer<T, P> implements Gatherer<T, Set<P>, T> {

   private final Function<T, P> selector;

   public DistinctByGatherer(Function<T, P> selector) {
       this.selector = selector;
   }

   @Override
   public Supplier<Set<P>> initializer() {
       return HashSet::new;
   }

   @Override
   public Integrator<Set<P>, T, T> integrator() {
       return Integrator.ofGreedy((state, item, downstream) -> {
           P extracted = selector.apply(item);
           if(!state.contains(extracted)) {
               state.add(extracted);
               downstream.push(item);
           }

           return true;
       });
   }
}

Here, the DistinctByGatherer relies on:

  • An initializer that sets up the internal state – namely, a set keeping track of the elements already seen.
  • An integrator that guarantees element uniqueness according to the given selector function. It works with:
    • state: Holds the internal set, created by the initializer() function, used to track unique elements.
    • item: The element of the input stream currently being processed.
    • downstream: The next stage of the pipeline, which receives the element if it passes the uniqueness check.

For easier use, we add a factory class, MyGatherers:

public class MyGatherers {
   public static <T, P> DistinctByGatherer<T, P> distinctBy(Function<T, P> extractor) {
       return new DistinctByGatherer<>(extractor);
   }
}

Applying this gatherer to the Money class:

var money = List.of(
       new Money(BigDecimal.valueOf(12), PLN),
       new Money(BigDecimal.valueOf(11), EUR),
       new Money(BigDecimal.valueOf(15), PLN)
);

System.out.println("\nDistinct By Currency:");
money.stream()
       .gather(MyGatherers.distinctBy(Money::currency))
       .forEach(System.out::println);

Output:

Distinct By Currency:
Money[amount=12, currency=PLN]
Money[amount=11, currency=EUR]

The DistinctByGatherer is a refined extension of the standard Stream::distinct method, providing more flexibility and richer filtering based on custom criteria rather than plain object equality.

Reduce by Gatherer

Now let’s look at the ReduceByGatherer, which aggregates stream elements according to a selector function. In contrast to the DistinctByGatherer, it doesn’t skip duplicates but merges elements that appear more than once.

public class ReduceByGatherer<T, P> implements Gatherer<T, Map<P, T>, T> {
   private final Function<T, P> selector;
   private final BiFunction<T, T, T> operation;

   public ReduceByGatherer(Function<T, P> extractor, BiFunction<T, T, T> reducer) {
       this.selector = extractor;
       this.operation = reducer;
   }

   @Override
   public Supplier<Map<P, T>> initializer() {
       return HashMap::new;
   }

   @Override
   public Integrator<Map<P, T>, T, T> integrator() {
       return Integrator.ofGreedy((state, item, _) -> {
           state.merge(selector.apply(item), item, operation);
           return true;
       });
   }

   @Override
   public BiConsumer<Map<P, T>, Downstream<? super T>> finisher() {
       return (state, downstream) -> state.values().forEach(downstream::push);
   }
}

Below is a revised, more thorough version of the section describing the key aspects of the ReduceByGatherer:

Key aspects of ReduceByGatherer:

  • State Management: Uses a HashMap to hold its state, which allows stream elements to be aggregated effectively.
  • Integrator Implementation: Merges elements as the stream is processed, following the given selector and reducer functions. Rather than sending processed elements straight downstream as in the earlier examples, this method has to keep them in a HashMap for state management.
  • Finisher Implementation: Completes the process by pushing the accumulated values downstream once every element has been handled. This is essential for reduction operations, in which all elements must be processed together before a final result can be produced.
  • Complete Stream Processing: In contrast to operations that allow early termination (such as takeWhile), ReduceByGatherer has to process every stream element to produce correct results.

Adding ReduceByGatherer to our utility class:

public class MyGatherers {
   public static <T, P> DistinctByGatherer<T, P> distinctBy(Function<T, P> extractor) {
       return new DistinctByGatherer<>(extractor);
   }

   public static <T, P> ReduceByGatherer<T, P> reduceBy(Function<T, P> extractor, BiFunction<T, T, T> reducer) {
       return new ReduceByGatherer<>(extractor, reducer);
   }
}

A sample use with the Money class:

var money = List.of(
       new Money(BigDecimal.valueOf(12), PLN),
       new Money(BigDecimal.valueOf(11), EUR),
       new Money(BigDecimal.valueOf(15), PLN)
);

System.out.println("Reduce By Currency");
money.stream()
       .gather(MyGatherers.reduceBy(Money::currency, Money::add))
       .forEach(System.out::println);

Output:

Reduce By Currency
Money[amount=27, currency=PLN]
Money[amount=11, currency=EUR]

This gatherer merges stream elements efficiently, which makes it a great fit for compacting or aggregating data in a finite stream.

Max by Gatherer

The MaxByGatherer finds the largest element in a stream according to a given selector function. Its implementation is more involved, since it adds a separate class for state management and a combiner function that improves its support for parallel stream operations.

class MaxByGatherer<T, B extends Comparable<B>> implements Gatherer<T, State<T>, T> {
   private final Function<T, B> extractor;

   MaxByGatherer(Function<T, B> extractor) {
       this.extractor = extractor;
   }

   @Override
   public Supplier<State<T>> initializer() {
       return State::new;
   }

   @Override
   public Integrator<State<T>, T, T> integrator() {
       return Integrator.ofGreedy((state, item, _) -> {
           if (state.maxElement == null) {
               state.maxElement = item;
               return true;
           }

           B currentItemValue = extractor.apply(item);
           B maxItemValue = extractor.apply(state.maxElement);

           if (currentItemValue.compareTo(maxItemValue) > 0) {
               state.maxElement = item;
           }

           return true;
       });
   }

   @Override
   public BinaryOperator<State<T>> combiner() {

       return (first, second) -> {
           if (first.maxElement == null && second.maxElement == null) {
               return null;
           } else if (first.maxElement == null) {
               return second;
           } else if (second.maxElement == null) {
               return first;
           }

           B firstMaxValue = extractor.apply(first.maxElement);
           B secondMaxValue = extractor.apply(second.maxElement);

           if (firstMaxValue.compareTo(secondMaxValue) > 0) {
               return first;
           } else {
               return second;
           }
       };
   }

   @Override
   public BiConsumer<State<T>, Downstream<? super T>> finisher() {
       return (state, downstream) -> downstream.push(state.maxElement);
   }
}

class State<T> {
   T maxElement;
}

Key Features:

  • State Management: Keeps track of the largest element with a custom State class.
  • Integrator: Finds the largest element using the supplied comparator function.
  • Combiner: Merges state from different segments, making processing in parallel streams efficient.

This gatherer considerably extends the Stream API’s capacity for complex computations, such as finding maximum values when processing in parallel.

Adding MaxByGatherer to MyGatherers:

public class MyGatherers {
   public static <T, P> DistinctByGatherer<T, P> distinctBy(Function<T, P> extractor) {
       return new DistinctByGatherer<>(extractor);
   }

   public static <T, P> ReduceByGatherer<T, P> reduceBy(Function<T, P> extractor, BiFunction<T, T, T> reducer) {
       return new ReduceByGatherer<>(extractor, reducer);
   }

   public static <T, B extends Comparable<B>> MaxByGatherer<T, B> maxBy(Function<T, B> extractor) {
       return new MaxByGatherer<>(extractor);
   }
}

Sample Usage:

var money = List.of(
       new Money(BigDecimal.valueOf(12), PLN),
       new Money(BigDecimal.valueOf(11), PLN),
       new Money(BigDecimal.valueOf(15), PLN)
);

System.out.println("\nMax By Amount");
money.stream()
       .parallel()
       .gather(MyGatherers.maxBy(Money::amount))
       .forEach(System.out::println);

Note that the combiner implementation only comes into play when we use a parallel stream.

Output:

Max By Amount
Money[amount=15, currency=PLN]

This implementation demonstrates how MaxByGatherer can find the maximum value in a collection efficiently.

Map Not-Null Gatherer

Imagine we have a list of Money objects interspersed with null values. We want to drop the nulls and transform the remaining elements. This could be done by combining Stream::filter and Stream::map, but let’s try a more streamlined approach modeled on Kotlin’s mapNotNull function.

public class MapNotNullGatherer<T,M> implements Gatherer<T, T, M> {
   private final Function<T, M> mapper;

   public MapNotNullGatherer(Function<T, M> mapper) {
       this.mapper = mapper;
   }

   @Override
   public Integrator<T, T, M> integrator() {
       return Integrator.ofGreedy((_, item, downstream) -> {
           if(item != null) {
               downstream.push(mapper.apply(item));
           }
           return true;
       });
   }
}

Key Highlights:

  • Simplicity: The only method this gatherer needs is the integrator, which drops nulls and applies the mapping function to the non-null elements.
  • Efficiency: Filtering and mapping happen in a single step, which improves both code readability and performance.

Adding MapNotNullGatherer to MyGatherers:

public class MyGatherers {
   public static <T, P> DistinctByGatherer<T, P> distinctBy(Function<T, P> extractor) {
       return new DistinctByGatherer<>(extractor);
   }

   public static <T, P> ReduceByGatherer<T, P> reduceBy(Function<T, P> extractor, BiFunction<T, T, T> reducer) {
       return new ReduceByGatherer<>(extractor, reducer);
   }

   public static <T, B extends Comparable<B>> MaxByGatherer<T, B> maxBy(Function<T, B> extractor) {
       return new MaxByGatherer<>(extractor);
   }

   public static <T, M> MapNotNullGatherer<T, M> mapNotNull(Function<T, M> mapper) {
       return new MapNotNullGatherer<>(mapper);
   }
}

Example of Use:

var moneyWithNulls = Arrays.asList(
       new Money(BigDecimal.valueOf(12), PLN),
       null,
       new Money(BigDecimal.valueOf(11), EUR),
       new Money(BigDecimal.valueOf(15), PLN),
       null
);

System.out.println("Map not-null Gatherer");
moneyWithNulls.stream()
       .gather(MyGatherers.mapNotNull(m -> m.multiply(BigDecimal.TWO)))
       .forEach(System.out::println);

Output:

Map not-null Gatherer
Money[amount=24, currency=PLN]
Money[amount=22, currency=EUR]
Money[amount=30, currency=PLN]

This implementation shows how custom gatherers can make stream operations simpler and better, especially when both filtering and transformation are needed.

Find first Gatherer

The FindFirstGatherer is an inventive take on the standard Stream::findFirst method that differs from it in two key ways:

  • Stream Output: Whereas Stream::findFirst returns an Optional<T>, this gatherer produces a Stream<T> – either empty or containing a single element, depending on whether the predicate is satisfied.
  • Predicate-Based Filtering: It works with a predicate, processing elements selectively until it finds the first match.
public class FindFirstGatherer<T> implements Gatherer<T, T, T> {
   private final Predicate<T> predicate;

   public FindFirstGatherer(Predicate<T> predicate) {
       this.predicate = predicate;
   }

   @Override
   public Integrator<T, T, T> integrator() {
       return Integrator.ofGreedy((_, item, downstream) -> {
           if (predicate.test(item)) {
               downstream.push(item);
               return false;
           } else {
               return true;
           }
       });
   }
}

As soon as an item satisfying the predicate is found, the implementation stops processing any further elements. When that happens, the integrator returns false.

Adding FindFirstGatherer to MyGatherers:

public class MyGatherers {
   public static <T, P> DistinctByGatherer<T, P> distinctBy(Function<T, P> extractor) {
       return new DistinctByGatherer<>(extractor);
   }

   public static <T, P> ReduceByGatherer<T, P> reduceBy(Function<T, P> extractor, BiFunction<T, T, T> reducer) {
       return new ReduceByGatherer<>(extractor, reducer);
   }

   public static <T, B extends Comparable<B>> MaxByGatherer<T, B> maxBy(Function<T, B> extractor) {
       return new MaxByGatherer<>(extractor);
   }

   public static <T, M> MapNotNullGatherer<T, M> mapNotNull(Function<T, M> mapper) {
       return new MapNotNullGatherer<>(mapper);
   }

   public static <T> FindFirstGatherer<T> findFirst(Predicate<T> predicate) {
       return new FindFirstGatherer<>(predicate);
   }
}

Example of Use:

var money = List.of(
       new Money(BigDecimal.valueOf(12), PLN),
       new Money(BigDecimal.valueOf(11), PLN),
       new Money(BigDecimal.valueOf(15), PLN)
);

System.out.println("Find first Gatherer");
money.stream()
       .gather(MyGatherers.findFirst(m -> m.currency().equals(PLN)))
       .forEach(System.out::println);

Output:

Find first Gatherer
Money[amount=12, currency=PLN]

This gatherer offers an elegant answer for cases where we need just the first matching element, simplifying the process in a more functional style.

Summary

This article took a thorough look at JEP-461: Stream Gatherers, a preview feature added in Java 22. We started by breaking down its core concepts and then built a range of operations showing how useful it is in specific situations. These experiments make it evident that this new Stream API abstraction greatly improves the customizability, reusability, and overall clarity of stream operations – a welcome step forward for data processing in Java.

In the next article, we will unlock the full potential of Stream Gatherers by looking at the built-in gatherers and at how a Gatherer can be composed from other Gatherers.

Share your thoughts on this enhancement and on its chances of becoming a stable feature in upcoming Java releases. As I see it, it’s an exciting change that makes working with streams both more pleasant and more efficient.