Scala: How to loop over a collection with ‘for’ and ‘foreach’ (plus for loop translation)

This is an excerpt from the 1st Edition of the Scala Cookbook (partially modified for the internet). This is Recipe 3.1, “How to loop over a collection with for and foreach (and how a for loop is translated).”

Scala Problem

You want to iterate over the elements in a Scala collection, either to operate on each element in the collection, or to create a new collection from the existing collection.

Scala Solution

There are many ways to loop over Scala collections, including for loops, while loops, and collection methods like foreach, map, flatMap, and more. This solution focuses primarily on the for loop and foreach method.

Given a simple array:

val a = Array("apple", "banana", "orange")

I prefer to iterate over the array with the following for loop syntax, because it’s clean and easy to remember:

scala> for (e <- a) println(e)
apple
banana
orange

When your algorithm requires multiple lines, use the same for loop syntax, and perform your work in a block:

scala> for (e <- a) {
     |   // imagine this requires multiple lines
     |   val s = e.toUpperCase
     |   println(s)
     | }
APPLE
BANANA
ORANGE

Returning values from a for-loop

Those examples perform an operation using the elements in an array, but they don’t return a value you can use, such as a new array. In cases where you want to build a new collection from the input collection, use the for/yield combination:

scala> val newArray = for (e <- a) yield e.toUpperCase
newArray: Array[java.lang.String] = Array(APPLE, BANANA, ORANGE)

The for/yield construct returns a value, so in this case, the array newArray contains uppercase versions of the three strings in the initial array. Notice that an input Array yields an Array (and not something else, like a Vector).

When your algorithm requires multiple lines of code, perform the work in a block after the yield keyword:

scala> val newArray = for (e <- a) yield {
     |   // imagine this requires multiple lines
     |   val s = e.toUpperCase
     |   s
     | }
newArray: Array[java.lang.String] = Array(APPLE, BANANA, ORANGE)

for-loop counters

If you need access to a counter inside a for loop, use one of the following approaches. First, you can access array elements with a counter like this:

for (i <- 0 until a.length) {
    println(s"$i is ${a(i)}")
}

That loops yields this output:

0 is apple
1 is banana
2 is orange

Scala collections also offer a zipWithIndex method that you can use to create a loop counter:

scala> for ((e, count) <- a.zipWithIndex) {
     |   println(s"$count is $e")
     | }
0 is apple
1 is banana
2 is orange

See Recipe 10.11, “Using zipWithIndex or zip to Create Loop Counters”, for more examples of how to use zipWithIndex.

for-loop generators and guards

On a related note, the following example shows how to use a Range to execute a loop three times:

scala> for (i <- 1 to 3) println(i)
1
2
3

The 1 to 3 portion of the loop creates a Range, as shown in the REPL:

scala> 1 to 3
res0: scala.collection.immutable.Range.Inclusive = Range(1, 2, 3)

Using a Range like this is known as using a generator. The next recipe demonstrates how to use this technique to create multiple loop counters.

Recipe 3.3 demonstrates how to use guards (if statements in for loops), but here’s a quick preview:

scala> for (i <- 1 to 10 if i < 4) println(i)
1
2
3

How to loop over a Map

When iterating over keys and values in a Map, I find this to be the most concise and readable for loop:

val names = Map("fname" -> "Robert", "lname" -> "Goren")

for ((k,v) <- names) println(s"key: $k, value: $v")

See Recipe 11.18, “Traversing a Map” for more examples of how to iterate over the elements in a Map.

Discussion

An important lesson from the for loop examples is that when you use the for/yield combination with a collection, you’re building and returning a new collection, but when you use a for loop without yield, you’re just operating on each element in the collection — you’re not creating a new collection. The for/yield combination is referred to as a for-comprehension, and in its basic use, it works just like the map method. It’s discussed in more detail in Recipe 3.4, “Creating a For Comprehension (for/yield Combination)”.

In some ways Scala reminds me of the Perl slogan, “There’s more than one way to do it,” and iterating over a collection provides some great examples of this. With the wealth of methods that are available on collections, it’s important to note that a for loop may not even be the best approach to a particular problem; the methods foreach, map, flatMap, collect, reduce, etc., can often be used to solve your problem without requiring an explicit for loop.

For example, when you’re working with a collection, you can also iterate over each element by calling the foreach method on the collection:

scala> a.foreach(println)
apple
banana
orange

When you have an algorithm you want to run on each element in the collection, just use the anonymous function syntax:

scala> a.foreach(e => println(e.toUpperCase))
APPLE
BANANA
ORANGE

As before, if your algorithm requires multiple lines, perform your work in a block:

scala> a.foreach { e =>
     |     val s = e.toUpperCase
     |     println(s)
     | }
APPLE
BANANA
ORANGE

Bonus: How Scala ‘for’ loops are translated by the compiler

As you work with Scala, it’s helpful to understand how for loops are translated by the compiler. The Scala Language Specification provides details on precisely how a for loop is translated under various conditions. I encourage you to read the Specification for details on the rules, but a simplification of those rules can be stated as follows:

1. A simple for loop that iterates over a collection is translated to a foreach method call on the collection.
2. A for loop with a guard (see Recipe 3.3) is translated to a sequence of a withFilter method call on the collection followed by a foreach call.
3. A for loop with a yield expression is translated to a map method call on the collection.
4. A for loop with a yield expression and a guard is translated to a withFilter method call on the collection, followed by a map method call.

Again, the Specification is more detailed than this, but those statements will help get you started in the right direction.

These statements can be demonstrated with a series of examples. Each of the following examples starts with a for loop, and the code in each example will be compiled with the following scalac command:

$ scalac -Xprint:parse Main.scala

This command provides some initial output about how the Scala compiler translates the for loops into other code.

As a first example, start with the following code in a file named Main.scala:

class Main {
    for (i <- 1 to 10) println(i)
}

This code is intentionally small and trivial so you can see how the for loop is translated by the compiler.

When you compile this code with the scalac -Xprint:parse command, the full output looks like this:

$ scalac -Xprint:parse Main.scala
[[syntax trees at end of parser]] // Main.scala
package <empty> {
  class Main extends scala.AnyRef {
    def <init>() = {
      super.<init>();
      ()
    };
    1.to(10).foreach(((i) => println(i)))
  }
}

For this example, the important part of the output is the area that shows the for loop was translated by the compiler into the following code:

1.to(10).foreach(((i) => println(i)))

As you can see, the Scala compiler translates a simple for loop over a collection into a foreach method call on the collection.

If you compile the file with the -Xprint:all option instead of -Xprint:parse, you’ll see that the code is further translated into the following code:

scala.this.Predef.intWrapper(1).to(10).foreach[Unit]
  (((i: Int) => scala.this.Predef.println(i)))

The code continues to get more and more detailed as the compiler phases continue, but for this demonstration, only the first step in the translation process is necessary.

Note that although I use a Range in these examples, the compiler behaves similarly for other collections. For example, if I replace the Range in the previous example with a List, like this:

// original List code
val nums = List(1,2,3)
for (i <- nums) println(i)

the for loop is still converted by the compiler into a foreach method call:

// translation performed by the compiler
nums.foreach(((i) => println(i)))

Given this introduction, the following series of examples demonstrates how various for loops are translated by the Scala 2.10 compiler. Here’s the first example again, showing both the input code I wrote and the output code from the compiler:

// #1 - input (my code)
for (i <- 1 to 10) println(i)

// #1 - compiler output
1.to(10).foreach(((i) => println(i)))

Next, I’ll use the same for loop but add a guard condition (an if statement) to it:

// #2 - input code
for {
    i <- 1 to 10
    if i % 2 == 0
} println(i)

// #2 - translated output
1.to(10).withFilter(((i) => i.$percent(2).$eq$eq(0))).foreach(((i) =>  println(i)))

As shown, a simple, single guard is translated into a withFilter method call on the collection, followed by a foreach call.

The same for loop with two guards is translated into two withFilter calls:

// #3 - input code
for {
    i <- 1 to 10
    if i != 1
    if i % 2 == 0
} println(i)

// #3 - translated output
1.to(10).withFilter(((i) => i.$bang$eq(1)))
        .withFilter(((i)
  => i.$percent(2).$eq$eq(0))).foreach(((i) => println(i)))

Next, I’ll add a yield statement to the initial for loop:

// #4 - input code
for { i <- 1 to 10 } yield i

// #4 - output
1.to(10).map(((i) => i))

As shown, when a yield statement is used, the compiler translates the for/yield code into a map method call on the collection.

Here’s the same for/yield combination with a guard added in:

// #5 - input code (for loop, guard, and yield)
for {
    i <- 1 to 10
    if i % 2 == 0
} yield i

// #5 - translated code
1.to(10).withFilter(((i) => i.$percent(2).$eq$eq(0))).map(((i) => i))

As in the previous examples, the guard is translated into a withFilter method call, and the for/yield code is translated into a map method call.

These examples demonstrate how the translations are made by the Scala compiler, and I encourage you to create your own examples to see how they’re translated by the compiler into other code. The -Xprint:parse option shows a small amount of compiler output, while the -Xprint:all option produces hundreds of lines of output for some of these examples, showing all the steps in the compilation process.

For more details, see the Scala Language Specification for exact rules on the for loop translation process. The details are currently in Section 6.19, “For Comprehensions and For Loops,” of the Specification.