Bloop’s compiler performance is ~29% faster than SBT (on at least one project)

I had read that Bloop was faster than Scala compiler tools like scalac and fsc, so I wondered if it was faster than SBT, and if so, how much faster. So I downloaded Eric Torreborre’s specs2 source code, which has 880 source code files, and compiled the entire project with both SBT and Bloop.

SBT performance

To test SBT’s speed, I ran all the commands from inside the SBT command prompt, which I usually do anyway to avoid the SBT/JVM startup lag time. I also ran clean and compile several times before recording SBT’s speed, because I thought that would be a better reflection of real-world usage and performance. I ran the tests four times, and the average time with SBT was 49 seconds, and that was very consistent, always coming in between 48 and 50 seconds.

Bloop performance

I did the equivalent thing with Bloop, starting its server and then running clean and compile several times before recording its performance four times. Bloop came in at 35 seconds. These times were also very consistent.

Result

Summary: Bloop was 14 seconds faster than SBT, for a 29% performance improvement.

Discussion

Note that if Bloop was 13 seconds faster that would be a 27% improvement, so one second makes a little difference to the percentage. I mention that because I didn’t record values after the decimal, so it’s possible that the difference may have been closer to 13.5 seconds, though it’s also possible that it was closer to 14.5 seconds. I should also note that I used SBT’s output time for its values, and the Unix time command to measure Bloop’s performance.

It’s also worth nothing that initially there was more variance in the times, with SBT coming in as high as 62 seconds, so I turned off my WiFi, shut everything down that I could shut down, and ran the tests again, and the times were much more consistent.

If you haven’t used Bloop before and want to try the same tests yourself, just follow the Bloop installation instructions, and then follow the “Export your build” steps for SBT in Step 4. After that, the Bloop usage docs are helpful.

I was curious how much faster Bloop is than SBT, and after confirming that the two compilers were doing the same work, it looks like Bloop is about 29% faster, at least on this one project.