Scala - How to download URL contents to a String or file

Scala URL FAQ: How do I download the contents of a URL to a String or file in Scala?

I ran a few tests last night in the Scala REPL to see if I could think of different ways to download the contents of a URL to a String or file in Scala, and came up with a couple of different solutions, which I'll share here.

Download URL contents to a String in Scala

The best way I could think of to download the contents of a URL to a String looks like this:

import scala.io.Source
val html = Source.fromURL("http://google.com")
val s = html.mkString
println(s)

If you really want to impress your friends and relatives, you can shorten that to one line, like this:

println(scala.io.Source.fromURL("http://google.com").mkString)

That seems like the most direct and "Scala like" approach to the problem.

Another approach is to reach out to the shell and use the wget or curl command, like this:

import sys.process._
val s = "curl http://google.com" !!

While that also works, I don't like depending on wget or curl when you can do everything you need from within the Scala/Java environment.

Download the contents of a URL to a file

If you want to download the contents of a URL directly to a file, you can do so like this:

import sys.process._
import java.net.URL
import java.io.File
new URL("http://google.com") #> new File("Output.html") !!

As you can see, most of that code involves importing the packages you need, and then the last line of code actually downloads the URL contents and saves it to a file named Output.html.

If you're not used to the syntax of that last line, please see my tutorial on how to execute system commands in Scala.

Other ideas?

I just wrote these ideas down fairly quickly last night while thinking about the problem, and there may be other and better ways to download UTL contents in Scala. If you know of better solutions, feel free to leave a note in the Comments section below.

Add new comment

The content of this field is kept private and will not be shown publicly.

Anonymous format

  • Allowed HTML tags: <em> <strong> <cite> <code> <ul type> <ol start type> <li> <pre>
  • Lines and paragraphs break automatically.
By submitting this form, you accept the Mollom privacy policy.