Collect a pipeline into what you need

Nothing in a pipeline runs until its terminal operation pulls the values through. Most of the time the result you need already has a name on Iterate itself; collect with an Accumulator is what is left for everything that does not.

Steps#

1. Reach for the named terminal operation first. toList(), toSet(), to<Target>() for any other From<Iterate<Item>>, sum(), count(), joined(separator:), fold(initial, combine), first(), find { ... }, any { ... }, all { ... } and groupBy { ... } cover most pipelines without ever naming an Accumulator.

   const total = orders.map { order => order.amount }.sum()
   const byCity = customers.groupBy { customer => customer.city }

2. Reach for collect with a standard-library Accumulator for a shape none of those name. counting(), summing { ... }, averaging { ... }, minBy { ... }/maxBy { ... }, partitioningBy { ... }, and groupingBy { ... }.then(downstream) for one collector per group.

   const stats = orders.collect(groupingBy { order: Order => order.city }.then(summing { order: Order => order.amount }))

3. Run the same accumulator more than once when several results come from one pipeline of values. An Accumulator is a value; each call to collect fills a copy of its own.

   const amounts = summing { order: Order => order.amount }
   const thisMonth = orders.filter { order => order.isThisMonth() }.collect(amounts)
   const lastMonth = orders.filter { order => order.isLastMonth() }.collect(amounts)

4. Write collector(initial, finish:, step:) when the state a run needs is one value. It answers an Accumulator whose accumulator folds step over every value and applies finish once, which covers most custom aggregates without a type of their own.

5. Write a type with var fields that implements Accumulator when the state is more than one value, the way the standard library's own groupingBy(...).then(...) keeps one accumulator per group in a Map.

Pitfalls#

• isDone() is what lets a terminal operation stop early. first(), find { ... } and collect behind take(n) never read a source to the end; a custom Accumulator that never overrides isDone() is asked to read everything, which is correct for a sum or a count and wrong for anything that could stop sooner.
• An Accumulator value is not itself a result. orders.collect(summing { order: Order => order.amount }) runs it; the value on its own is a description that can be handed to collect again for a different source.
• fold and a hand-written Accumulator do the same pull. Reach for fold when the state is a simple running value and there is only one call site; reach for a named Accumulator the moment two pipelines need the same aggregate, so the logic is written once.
• A collection is not an Accumulator. pipeline.collect(existingList) is not a thing: name the accumulator that gathers into one - listing() or ListAccumulator<Item>() for a list, into<Set<Int>>() for anything with From<Iterate<Item>> - and append the result to what you already have.

Full example#

type Order {
  city: String
  amount: Int
}

const orders = [
  Order("Berlin", 120),
  Order("Berlin", 80),
  Order("Munich", 200),
]

const total = orders.map { order => order.amount }.sum()
print total

const totalsByCity = orders.collect(groupingBy { order: Order => order.city }.then(summing { order: Order => order.amount }))
print totalsByCity

const highValue = orders.collect(partitioningBy { order => order.amount >= 100 })
print highValue

Related#

• Pipelines - source, lazy stages and terminal operation, in full.
• Collectors - Accumulator and every standard collector in one place.
• Sort by more than one key - a sorted stage in front of the same kind of pipeline.