NOTE: If you read my previous article (a Scala functional programming style To-Do List application), the new content in this article starts down at The Scala/FP Code section.
Back when I was writing Functional Programming, Simplified I started to write a little Scala/FP “To-Do List” application that you can run from the command line, based on a similar application in the Learn You A Haskell For Great Good book. For reasons I don’t remember, I decided not to include it in the book, and forgot about it until I started using GraalVM (“Graal”) recently.
Graal includes a native image feature lets you compile JVM classes and JAR files into native executables, so as I thought about things I can make faster, I was reminded of the To-Do List app and thought about how cool it would be if it started instantaneously. So I found the old project, blew the dust off of it (updated all of its dependencies), and made a few additions so I could create (a) a single, executable JAR file with sbt-assembly, and (b) a native executable with Graal.
Because I wrote the application for my book, I wrote the code using Scala in a functional programming (FP) style. If you read the book, the code should look very familiar, with the main exception being that I use some tools from the Cats project.
How the application works
As mentioned, this is a command-line To-Do List application, and here’s a short demonstration of how it, starting with the v command (which stands for “view”) to show that there’s nothing in the initial list, followed by some add commands and a rm command:
Command ('h' for help, 'q' to quit)
==> h
Possible commands
-----------------
add <task> - add a to-do item
h - show this help text
rm [task number] - remove a task by its number
v - view the list of tasks
q - quit
> ./todo
Command ('h' for help, 'q' to quit)
==> v
Command ('h' for help, 'q' to quit)
==> add wake up
1. wake up
Command ('h' for help, 'q' to quit)
==> add make coffee
1. wake up
2. make coffee
Command ('h' for help, 'q' to quit)
==> add go to work
1. wake up
2. make coffee
3. go to work
Command ('h' for help, 'q' to quit)
==> rm 3
1. wake up
2. make coffee
The Scala/FP code
If you’re interested in functional programming with Scala, the source code is available at this URL:
Here’s a short overview of how the code works:
ToDoListFIO extends Appis where the action starts.- As its name implies,
mainLoopis the application’s main loop. It runs until you enterq. - This line of code in the loop converts the user’s input into a
Command:scala cmd <- getLine.map(Command.parse _) - Commands are then processed in the
processCommandmethod.
Cats’ unsafeRunSync
Something new you’ll see in the code is a call to the Cats Scalaz’s unsafeRunSync function. The Scaladoc for unsafeRunSync states:
Produces the result by running the encapsulated effects as impure side effects.
As the name says, this is an UNSAFE function as it is impure and performs side effects, not to mention blocking, throwing exceptions, and doing other things that are at odds with reasonable software. You should ideally only call this function *once*, at the very end of your program.
Another way to state that is that with Cats you write all of your I/O code using its IO monad, and then make a single call to unsafeRunSync when you’re ready to trigger the execution of your application. In fact, if you don’t call unsafeRunSync, your application won’t do anything. (Side effects are a very big deal in functional programming, hence the scary “unsafe” method name.)
Thanks to (a) Eric Torreborre and (b) some good Cats documentation, the code for this Cats To-Do List application is improved over the similar code I wrote using Scalaz. Most of the code in ToDoListFIO.scala is significantly better than the code in the original Scalaz version of this app. Here’s the new main loop:
def mainLoop: IO[Unit] = for {
_ <- IOFunctions.putStr(prompt)
cmd <- getLine.map(Command.parse _)
_ <- if (cmd == Quit) {
IO.unit
} else {
processCommand(cmd) >> mainLoop
}
} yield ()And here’s the new and improved view method:
def view: IO[Unit] = for {
lines <- readFile(datafile)
result <- IO((for ((line,i) <- lines.zip(Stream from 1)) yield s"$i. $line").mkString("\n"))
_ <- putStrLn(result + "\n")
} yield ()
If you’re interested in the details about how this works, I encourage you to read the cats-effect tutorial. I love good documentation, and it helped me to understand their IO implementation. Because of the documentation, when I have some more free time I’ll also update the To-Do List app to use their IOApp rather than the built-in Scala App.
Update: Automatically releasing resources with Cats `bracket`
As a brief update, I created readFile and writeFile functions using the Cats bracket function, which follows an acquire/use/release pattern:
// `bracket`
def readFile(filename: String): IO[Seq[String]] = {
// acquire
IO(Source.fromFile(filename)).bracket { source =>
// use
IO((for (line <- source.getLines) yield line).toVector)
} { source =>
// release
IO(source.close())
}
}
/**
* Use Cats `bracket`:
* https://typelevel.org/cats-effect/tutorial/tutorial.html#what-about-bracket
*/
def writeFile(filename: String, text: String, append: Boolean): IO[Unit] = IO {
// acquire
IO(new BufferedWriter(new FileWriter(new File(filename), append))).bracket { bw =>
// use
IO(bw.write(text + "\n"))
} { bw =>
// release (note that you can put whatever logic you want here, such as logging)
IO(bw.close())
}.unsafeRunSync()
}Note that this is similar to making sure that you close the resource in a finally block after opening it in a try block.
At the moment I don’t know why the writeFile function requires a call to unsafeRunSync(), but I can confirm that it doesn’t do anything unless that’s called.
The Cats bracket documentation also has a little information about error handling, i.e., what to do if there is a problem reading from or writing to the file.
Update: Automatically releasing resources with Cats Resource
I also created second versions of readFile and writeFile using the Cats Resource, which is also used for resource management:
// `Resource`
def readFile(filename: String): IO[Seq[String]] = {
val acquire: IO[BufferedSource] = IO(Source.fromFile(filename))
Resource.fromAutoCloseable(acquire)
.use { source =>
IO {
val lines = (for (line <- source.getLines) yield line).toVector
IO(lines)
}
}
.unsafeRunSync()
}
/**
* Use cats `Resource`
* Definitely need `unsafeRunSync()` with Resource:
* https://typelevel.org/cats-effect/datatypes/resource.html
*/
def writeFile(filename: String, text: String, append: Boolean): IO[Unit] = IO {
val acquire = IO {
new BufferedWriter(new FileWriter(new File(filename), append))
}
// BufferedWriter extends java.lang.AutoCloseable, so this works
Resource.fromAutoCloseable(acquire)
.use(bw => IO(bw.write(text + "\n")))
.unsafeRunSync()
}Note in the last comment that fromAutoCloseable works because BufferedWriter extends java.lang.AutoCloseable.
In this case the Cats Resource documentation clearly shows that unsafeRunSync() is required when using Resource (implying that it is lazily evaluated).
Hopefully the rest of the code makes sense
If you read my FP book, hopefully the rest of the code will make sense. All of the I/O functions use an IO monad, and the rest of the code consists of pure functions.
The Cats documentation
The Typelevel people behind Cats and Cats-Effects have created some nice documentation here:
- The Cats IO data type
- The Cats-Effect tutorial
- An IO monad for Cats (older, but still helpful)
Two bugs (oops)
The code (still) has two bugs that I haven’t fixed.
First, if the todo.dat file doesn’t exist, the “view” command will throw an exception. (You can create an empty initial file to avoid that bug, or fix the code.)
Second, I noticed that I hadn’t fixed a bug related to the “remove” process. If you type in something like this:
rm fooinstead of this:
rm 1the application will blow up. The remove function assumes that it receives an Int, but I don’t do anything to validate the user input.
I was going to fix these bugs, but I’m short on time, so I thought I’d leave them as an exercise for the interested reader. (My apologies that I’m a little rushed tonight.)
Creating a single, executable JAR file
To create a single, executable JAR file, use the sbt assembly command, or use the assembly command at the SBT prompt. That command should “just work” for you, but if you need more information, I wrote this sbt-assembly tutorial as part of the Scala Cookbook.
GraalVM
As mentioned, I got back into this project because of Graal. Over the last few weeks I’ve been updating some of my JVM-based shell scripts to use Graal so they’ll start faster, and for however my brain works, it triggered a, “Hey, you wrote a To-Do List app a few years ago” thought. Indeed, using Graal, the app starts up immediately after you type todo.
If you haven’t used Graal yet it takes a little work to set it up initially. Here’s a very short description of what it takes to create a native executable with Graal for this project:
- Install GraalVM on your system
- Configure your
JAVA_HOMEandPATHenvironment variables to use Graal cdinto this project’s Graal directory- In my case I don’t use Graal all the time, so I open a new MacOS Terminal tab where I only use Graal, then source the 1setup_graal file to set the necessary GraalVM parameters (i.e.,
. 1setup_graal) - After I run
sbt-assembly, I then run the 2compile_graal.sh script to create thetodonative image (executable)
While the app starts pretty fast with the java-jar command, it starts instantly when you create the Graal native image, and that’s a really cool feeling, especially if you’ve been sadly used to that initial Java/JVM startup lag time for 20 years.
Summary
In summary, if you wanted to see how to write a small but complete functional programming application in Scala using Cats and Cats-Effect — or if you just wanted a command-line To-Do List application — I hope this source code and project is helpful.
Finally, as a bit of self-promotion, if that code is hard to understand, please see my Functional Programming, Simplified book. About 40% of the book is available as a free preview, so hopefully you can get a good feel about whether or not you want to buy the book from that preview.
| this post is sponsored by my books: | |||
#1 New Release |
FP Best Seller |
Learn Scala 3 |
Learn FP Fast |
All the best,
Al
Update: Bonus: A look at `IO` and `runUnsafeSync`
A great feature about SBT that’s not shown in its help text is that you can start a Scala REPL from inside SBT, and it will have all of your classes and resources available to you. This gives you a simple way to experiment with things such as Cats’ runUnsafeSync. Here’s the output from a SBT/REPL session where I test my readFile function. First I start the console:
sbt:FPToDoListWithCats> console [info] Starting scala interpreter... Welcome to Scala 2.12.8
Then I import my IOFunctions:
scala> import todolist.IOFunctions._ import todolist.IOFunctions._
Now I run readFile on my /etc/passwd file:
scala> readFile("/etc/passwd")
res0: cats.effect.IO[Seq[String]] = IO$274228422
At this point nothing happens. This is because IO is lazily evaluated, meaning that it’s not going to do anything until you force it to. You force it to run by calling unsafeRunSync on the resulting variable, res0:
scala> res0.unsafeRunSync res1: Seq[String] = Vector(##, # User Database, "# ", # Note that this file is consulted directly only when the system is running, # in single-user mode. ...
Now I see the contents of /etc/passwd, and res1 is a Seq[String].
Next, try the same thing on a file that doesn’t exist:
scala> readFile("/etc/passwd-doh")
res2: cats.effect.IO[Seq[String]] = IO$1614653826
Notice again that nothing happens (because IO is lazy). Now this time when I call unsafeRunSync on res2, I get a FileNotFoundException because I intentionally messed up the input filename:
scala> res2.unsafeRunSync java.io.FileNotFoundException: /etc/passwd-doh (No such file or directory) at java.io.FileInputStream.open0(Native Method) at java.io.FileInputStream.open(FileInputStream.java:195) at java.io.FileInputStream.<init>(FileInputStream.java:138) at scala.io.Source$.fromFile(Source.scala:94) at scala.io.Source$.fromFile(Source.scala:79) at scala.io.Source$.fromFile(Source.scala:57) at todolist.IOFunctions$.$anonfun$readFile$1(IOFunctions.scala:65) at cats.effect.internals.IORunLoop$.cats$effect$internals$IORunLoop$$loop(IORunLoop.scala:87) at cats.effect.internals.IORunLoop$.start(IORunLoop.scala:34) at cats.effect.internals.IOBracket$.$anonfun$apply$1(IOBracket.scala:44) at cats.effect.internals.IOBracket$.$anonfun$apply$1$adapted(IOBracket.scala:34) at cats.effect.internals.IORunLoop$RestartCallback.start(IORunLoop.scala:341) at cats.effect.internals.IORunLoop$.cats$effect$internals$IORunLoop$$loop(IORunLoop.scala:119) at cats.effect.internals.IORunLoop$.start(IORunLoop.scala:34) at cats.effect.IO.unsafeRunAsync(IO.scala:258) at cats.effect.internals.IOPlatform$.unsafeResync(IOPlatform.scala:38) at cats.effect.IO.unsafeRunTimed(IO.scala:325) at cats.effect.IO.unsafeRunSync(IO.scala:240)
I hope this helps to demonstrate a little more about how IO and unsafeRunSync work, as well as how to start a console/REPL session inside SBT so you can easily test Cats, Cats-Effect, and my functions.
This post is sponsored by my new book,
Learn Functional Programming Without Fear.
books by alvin
|
|
|
|
