How to use multiple Futures in a Scala for-comprehension

If you want to create multiple Scala Futures and merge their results together to get a result in a for comprehension, the correct approach is to (a) first create the futures, (b) merge their results in a for comprehension, then (c) extract the result using onComplete or a similar technique.

The correct approach (simplified)

I show the correct approach to using multiple futures in a for-expression in the following code. The important point is to create your futures as shown in step “(a)”, prior to using them in the for-expression in step “(b)”:

package futures

import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
import scala.util.{Failure, Success}

/**
 * @author: alvin alexander, https://alvinalexander.com
 */
object MultipleFutures1 extends App {

    // (a) create the futures
    val f1 = Future { sleep(800); 1 }
    val f2 = Future { sleep(200); 2 }
    val f3 = Future { sleep(400); 3 }

    // (b) run them simultaneously in a for-comprehension
    val result = for {
        r1 <- f1
        r2 <- f2
        r3 <- f3
    } yield (r1 + r2 + r3)

    // (c) do whatever you need to do with the result
    result.onComplete {
       case Success(x) => println(s"\nresult = $x")
       case Failure(e) => e.printStackTrace
    }

    // important for a little parallel demo: keep the jvm alive
    sleep(3000)
    
    def sleep(time: Long): Unit = Thread.sleep(time)

}

A thorough example for verification

There’s no way to tell from that code that this is the correct approach, so if you don’t trust me, or just want to test this on your own, you can use the following complete “Futures + for-comprehension” example to show how this works:

package futures

import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
import scala.util.{Failure, Success}

/**
 * @author: alvin alexander, http://alvinalexander.com
 */
object MultipleFutures2 extends App {

    val startTime = time

    // Future #1
    println(s"creating f1:  $delta")
    val f1 = slowlyDouble(x=1, delay=1500, name="f1")
    
    // Future #2
    Thread.sleep(100)
    println(s"\ncreating f2:  $delta")
    val f2 = slowlyDouble(x=2, delay=250, name="f2")
    
    // Future #3
    Thread.sleep(100)
    println(s"\ncreating f3:  $delta")
    val f3 = slowlyDouble(x=3, delay=500, name="f3")

    println(s"\nentering `for`: $delta")
    val result = for {
        r1 <- f1
        r2 <- f2
        r3 <- f3
    } yield (r1 + r2 + r3)

    println("\nBEFORE onComplete")
    result.onComplete {
       case Success(x) => {
           println(s"\nresult = $x (delta = $delta)")
           println("note that you don't get the result until the last future completes")
       }
       case Failure(e) => e.printStackTrace
    }
    println("AFTER onComplete\n")

    // important for a little parallel demo: keep the jvm alive
    sleep(3000)
    
    def slowlyDouble(x: Int, delay: Int, name: String): Future[Int] = Future {
        println(s"entered $name:  $delta")
        sleep(delay)
        println(s"leaving $name:  $delta")
        x * 2
    }
   
    def delta = System.currentTimeMillis - startTime
    def time = System.currentTimeMillis
    def sleep(time: Long): Unit = Thread.sleep(time)

}

If you run that code you’ll see output that looks like this:

creating f1:  17
entered f1:  34

creating f2:  138
entered f2:  139

creating f3:  243

entering `for`: 243
entered f3:  243

BEFORE onComplete
AFTER onComplete

leaving f2:  394
leaving f3:  748
leaving f1:  1538

result = 12 (delta = 1541)
note that you don't get the result until the last future completes

The output shows several interesting points:

• f1, f2, and f3 begin running immediately. You can’t tell it from this code, but they immediately begin running on new threads.
• The output rapidly passes over the onComplete statement.
• After a short pause the “leaving” statements are printed, followed quickly by the result.
• Notice that the delta printed with the result is just a little larger than the delta for f1. That’s because f1 has the longest sleep time. It makes sense that the total run time of three futures running in parallel is just a little larger than the future with the longest run time.

I encourage you to play with that code and make it your own until you’re completely satisfied that you understand how Scala futures work with a for comprehension.

The wrong approach

You can also confirm that the previous approach is correct by doing the wrong thing. The following code shows the WRONG way to use futures in a Scala for comprehension:

package futures

import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
import scala.util.{Failure, Success}

/**
 * @author: alvin alexander, http://alvinalexander.com
 */
object MultipleFuturesWrong extends App {

    val startTime = time

    // THIS IS INTENTIONALLY WRONG
    println(s"\nentering `for`: $delta")
    val result = for {
        r1 <- slowlyDouble(x=1, delay=1500, name="f1")
        r2 <- slowlyDouble(x=2, delay=250, name="f2")
        r3 <- slowlyDouble(x=3, delay=500, name="f3")
    } yield (r1 + r2 + r3)

    println("\nBEFORE onComplete")
    result.onComplete {
       case Success(x) => {
           println(s"\nresult = $x (delta = $delta)")
           println("note that you don't get the result until the last future completes")
       }
       case Failure(e) => e.printStackTrace
    }
    println("AFTER onComplete\n")

    // important for a little parallel demo: keep the jvm alive
    sleep(3000)
    
    def slowlyDouble(x: Int, delay: Int, name: String): Future[Int] = Future {
        println(s"entered $name:  $delta")
        sleep(delay)
        println(s"leaving $name:  $delta")
        x * 2
    }
   
    def delta = System.currentTimeMillis - startTime
    def time = System.currentTimeMillis
    def sleep(time: Long): Unit = Thread.sleep(time)

}

When you run that code you’ll see output similar to this:

entering `for`: 17
entered f1:  33

BEFORE onComplete
AFTER onComplete

leaving f1:  1534
entered f2:  1535
leaving f2:  1789
entered f3:  1790
leaving f3:  2295

result = 12 (delta = 2295)
note that you don't get the result until the last future completes

That output shows:

• The f1 Future is quickly entered (at delta = 33).
• The f1 Future is exited 1500 ms later.
• f2 is entered at that time, and does not exit for more than 250 ms.
• After that, f3 is entered, and the code pauses for another 500+ ms.

This shows that f1, f2, and f3 are NOT run in parallel, but are instead run one after the other. To be clear, this is WRONG, and not what you want.

Summary

As a little summary, if you wanted to see how to use multiple Scala Futures in a for comprehension, I hope this is helpful. I tried to show the correct and incorrect approaches in a few simple examples.