Posts in the “scala” category

I don’t want to make my original Parsing Apache access log records with Spark and Scala article any longer, so I’m putting some new, better code here.

Assuming that you read that article, I’ll jump right in and say that I use this code to load my data into the Spark REPL:

Last week I wrote an Apache access log parser library in Scala to help me analyze my Apache HTTP access log file records using Apache Spark. The source code for that project is hosted here on Github. You can use this library to parse Apache access log “combined” records using Scala, Java, and other JVM-based programming languages.

My last edits to the Scala Cookbook were in June, 2013, and after all this time there aren’t many things I wish I had added to the Cookbook. Yesterday I ran into one thing that I don’t think I included in the Cookbook: How to process multiple Option values in a Scala for loop (for comprehension). Here’s a quick look at how to do this.

For the impatient

For those who just want to see a for comprehension that processes multiple input Option values, here you go:

When I wrote the Scala Cookbook, I gave each recipe and then each chapter my full attention. I thought that if I wrote each recipe as well as possible, and included important recipes in each chapter, well, I wanted each chapter to be worth the price of the entire book. That was my goal.

As a result of this effort -- and perhaps to the chagrin of my editor -- the Scala collections chapters ended up being 130 pages in length.

This short blog post contains a collection of Scala number and date examples. I created most of these in the process of writing the Scala Cookbook. Unlike the Cookbook, I don’t describe the examples here much at all, I just show the examples, mostly as a reference for myself (and anyone else that can benefit from them).

Scala numeric types

Scala has these numeric types:

This post contains a collection of Scala control structures examples. I initially created most of these in the process of writing the Scala Cookbook. Unlike the Cookbook, I don’t describe them much here, I just show the examples, mostly as a reference for myself (and anyone else that can benefit from them).

if/then control structures:

Here are some examples of the Scala if/then control structure:

This post contains a collection of examples of Scala classes and class properties. I created most of these in the process of writing the Scala Cookbook. Unlike the Cookbook, I don’t describe them much here, I just show the examples, mostly as a reference for myself (and anyone else that can benefit from them).

This page contains a collection of Scala trait examples. I created many of these examples when I was writing the Scala Cookbook. Unlike the Cookbook, where I explain these examples in great detail, on this page I’m just sharing many of the examples so you can use this as a trait reference page. (The Cookbook actually contains more examples than this page.)

Without any more introduction, here are the examples.

This page contains a collection of Scala method examples. I created many of these examples while I was writing the Scala Cookbook.

Unlike the Cookbook, where I explain these examples in great detail, on this page I’m just sharing many of the examples so you can use this as a method/function reference page. (The Cookbook contains more examples than this page, and explains them in detail.)

This article is a collection of Scala “object” examples. I put the word object in quotes there because it has at least two meanings in Scala. In the first meaning, just like Java, an object is an instance of a class.

In its second meaning, Scala has an object keyword, and using that keyword lets you do a variety of things, including creating a main method to launch your application, to create the equivalent of Java’s static methods, and also to create something called a companion object.

As an experiment — that’s a big emphasis on the word “experiment” there — I decided to create an @impure annotation for my current Scala project. The annotation is a “do nothing” annotation, so it doesn’t actually check the code in the way the @tailrec annotation works, for example.

This page contains examples of methods that are available on Scala sequential collections, i.e., List, Array, Vector, ArrayBuffer, and sequential collections.

Symbolic method names

The first examples will show how to use sequence methods whose names are like ++ , ++:, and so on. First, we’ll create two sample lists:

val evens = List(2, 4, 6)
val odds = List(1, 3, 5)

And here are examples of how to use these methods whose names are just symbols:

I’ve written a variety of small Scala apps that take advantage of the “text to speech” capabilities on Mac OS X (Sarah, Wikipedia Page Reader), and a few days ago I started thinking about consolidating these by creating a Mac “text-to-speech service.” I initially created that as an Akka server (here on Github), then thought to make it a little more generic as a REST web service.

Scala FAQ: How can I iterate through each character in a Scala String, performing an operation on each character as I traverse the string?

Solution

Depending on your needs and preferences, you can use the Scala map or foreach methods, a for loop, or other approaches.
 

Scala String FAQ: How can you determine whether a String contains a regular expression pattern in Scala? (Or, “How can I find the first match (or all matches) of a regex in a String?”)

Solution: findFirstIn, findAllIn

Create a Regex object by invoking the .r method on a String, and then use that pattern with findFirstIn when you’re looking for one match, and findAllIn when looking for all matches.

Someone asked an interesting question this past week: What’s the difference between a Scala Future and a Thread?

At first I thought the answer was pretty obvious, but as I tried to explain it, I realized that I needed to consider the differences more before speaking. After some thinking, I finally decided that the differences are in their APIs, and as a result of that, significant differences in how they are used.

If you’re just interested in my summary/conclusions, jump down to the Summary section. Otherwise, read on.

Warning: This article is a work-in-progress. I need to put much more research into it.

Scala/Java Thread API

Technically there is no such thing as a “Scala Thread.” If you type this in your Scala IDE:

val t = new Thread

you’ll see that a Thread is really a java.lang.Thread. Putting that technicality behind us, some of the most important parts of the Java Thread API looks like this:

Constructors:

    Thread()
    Thread(Runnable target)

Methods:

    interrupt
    join
    run
    start
    yield

Here’s an example of how you might use a Thread in Scala:

val thread = new Thread {
    override def run {
        // put your long-running code here ...
    }
}
thread.start

As the API and code shows, a Thread is a general-purpose concurrency abstract. While you may not notice it at this point, it’s important to note that the Thread has no return type.

The Thread Javadoc gives this description of a Thread:

A thread is a thread of execution in a program. The Java Virtual Machine allows an application to have multiple threads of execution running concurrently.

Scala Future

The Scala Future is well-described on the official Futures and Promises page:

Futures provide a nice way to reason about performing many operations in parallel -- in an efficient and non-blocking way. The idea is simple; a Future is a sort of a placeholder object that you can create for a result that does not yet exist. Generally, the result of the Future is computed concurrently and can be later collected. Composing concurrent tasks in this way tends to result in faster, asynchronous, non-blocking parallel code.

I read somewhere else that a Future “represents the result of an asynchronous computation,” and I like that description.

They also add this:

A Future is an object holding a value which may become available at some point. (My emphasis.)

This value is usually the result of some other computation: (1) If the computation has not yet completed, we say that the Future is not completed. (2) If the computation has completed with a value or with an exception, we say that the Future is completed.

The Future API is significantly different from a Java Thread. I provide a Future example here that shows you can create a Future like this:

val f = Future {
    // your long-running task here that returns an Int ...
}

Note that a Future has a type, so I could have written that code like this to be more explicit:

val f: Future[Int] = Future {
    // your long-running task here that returns an Int ...
}

A common way to use a Future is to use its callback methods. You can do this in a variety of ways. One way is with the onComplete method, where you handle whether the Future returned successfully, or not:

f.onComplete {
    case Success(value) => println(s"Got the callback, meaning = $value")
    case Failure(e) => e.printStackTrace
}

You also also use the onSuccess and onFailure callback methods, like this:

f onSuccess {
    case 0 => println("got a zero")
}

and this:

f onFailure {
    case t => println("D'oh! Got an error: " + t.getMessage)
}

In some cases it can be inconvenient to use these methods, so the Scala Future “provides combinators which allow a more straightforward composition.” The “Futures and Promises” page shows this example of how to use a map method with a Future:

val rateQuote = future {
    connection.getCurrentValue(USD)
}

val purchase = rateQuote map { quote => 
    if (isProfitable(quote)) connection.buy(amount, quote)
    else throw new Exception("not profitable")
}

purchase onSuccess {
    case _ => println("Purchased " + amount + " USD")
}

I’m not going to go into the details of this example, I just want to show another way the API is different. Please see the Futures and Promises tutorial to read more about this example.

To summarize the Scala Future, its API looks like this:

Creating a Future:

    val f = Future {
        // your long-running code here ...
    }

Some of the methods available on a Future:

    isCompleted
    onComplete
    ready
    result
    value
    
    andThen
    collect
    filter
    flatMap
    foreach
    map
    onFailure
    onSuccess
    recover
    recoverWith
    zip

Finally, I think this text from the “Futures and Promises” page is also important:

We should now comment on when exactly the callback gets called. Since it requires the value in the future to be available, it can only be called after the future is completed. However, there is no guarantee it will be called by the thread that completed the future or the thread which created the callback. Instead, the callback is executed by some thread, at some time after the future object is completed. We say that the callback is executed eventually.

So, when you manually use a Thread, you wait for that one thread to return, but when you use a Future, there’s no guarantee that the callback will be called on the same thread that your Future ran on.

The Future as a ConcurrentTask

As I wrote in this article, when you first start working with a Future, it may help to think of it as a “concurrent task.” I demonstrated that in this code:

package futures

import scala.concurrent.{Future => ConcurrentTask}           // rename
import scala.concurrent.ExecutionContext.Implicits.global
import scala.util.{Failure, Success}

object FutureAsConcurrentTask extends App {

    // run some long-running task (task has type Future[Int] in this example)
    val task = ConcurrentTask {
        Cloud.executeLongRunningTask
    }

    // whenever the task completes, execute this code
    task.onComplete {
        case Success(value) => println(s"Got the callback, value = $value")
        case Failure(e) => println(s"D'oh! The task failed: ${e.getMessage}")
    }
}

In that code I renamed the Future class to ConcurrentTask when I imported it. At the time I wrote that article, that way of thinking helped me understand Scala futures.

Summary: Thread vs Future

I know that I need to write more about this and clarify my points, but as a summary for today:

• A Java Thread represents a thread of execution in an Java/Scala/JVM application.
• You can run code in parallel in a Thread, but a Thread does not have a return type.
• A Scala Future represents the result of an asynchronous computation, and has a return type.
• A Scala Future works with callback methods.
• A Scala Future can be composed, and has usual collection methods like map, flatMap, filter, etc.
• There is no guarantee that your Future’s callback method will be called on the same thread the future was run on.

Notes to self

As a “note to self,” I need to research and discuss how a thread works with the JVM’s shared memory model, and how the Scala Future compares to that.

Scala problem: You want to define an equals method for your class so you can compare object instances to each other.

Solution

If you’re new to Scala, a first thing to know is that object instances are compared with ==:

"foo" == "foo"   // true
"foo" == "bar"   // false
"foo" == null    // false
null == "foo"    // false
1 == 1           // true
1 == 2           // false
1d == 1.0d       // true

case class Person(name: String)
Person("Jess") == Person("Jessie")   // false

This is different than Java, which uses == for primitive values and equals for object comparisons.

Problem: You want to define a Scala method that takes a function as a parameter, and that function may have one or more input parameters, and may also return a value.

Solution: Following the approach described in the previous recipe, define a method that takes a function as a parameter. Specify the function signature you expect to receive, and then execute that function inside the body of the method.

This is an excerpt from the Scala Cookbook (partially modified for the internet). This is Recipe 13.9, “Simple concurrency with Scala Futures.”

Problem

You want a simple way to run one or more tasks concurrently in a Scala application, including a way to handle their results when the tasks finish. For instance, you may want to make several web service calls in parallel, and then work with their results after they all return.

Solution

A Future gives you a simple way to run an algorithm concurrently. A future starts running concurrently when you create it and returns a result at some point, well, in the future. In Scala, it’s said that a future returns “eventually.”

This is an excerpt from the 1st Edition of the Scala Cookbook (partially modified for the internet). This is Recipe 14.6, “How to disassemble and decompile Scala code.”

Problem

In the process of learning Scala, or trying to understand a particular problem, you want to examine the source code and bytecode the Scala compiler generates from your original source code.

Solution

You can use several different approaches to see how your Scala source code is translated: