ScalaTest 109: Mark your tests with tags to include or exclude them

By Alvin Alexander. Last updated: December 8, 2021

Problem: You want a way to label your individual ScalaTest unit tests, so you can easily choose to include or exclude them when running your test suite. For instance, you may want to tag long-running tests like database or web service tests, because they take a long time to run, and you don’t want to run them all the time.

Solution

Create one or more custom tags, then include those tags in your test specifications. When you run your tests, declare which tests you want to run, or not run.

To demonstrate this, begin with the Pizza and Topping classes shown in Writing TDD Tests with ScalaTest, and the BDD test class defined in Writing a First BDD Test with ScalaTest.

Next, add a new class in the src/test/scala/com/acme/pizza directory named Tags.scala, and add this content:

package com.acme.pizza

import org.scalatest.Tag

object DatabaseTest extends Tag("DatabaseTest")

This defines a Tag named DatabaseTest that you can use in your tests.

Next, add the tag to your database tests. This is how you add a tag to an it BDD-style test:

// add the 'DatabaseTest' tag to the 'it' method
it("Should start with no toppings", DatabaseTest) {
  Given("a new pizza")
  pizza = new Pizza
  Then("the topping count should be zero")
  assert(pizza.getToppings.size == 0)
}

Now when you run your tests, you can specify which tests you want to include. To include tests with the DatabaseTest tag in the list of tests to be run, start SBT, and then issue the following test-only command from within the SBT shell:

$ sbt

sbt> test-only -- -n DatabaseTest

To exclude tests with the DatabaseTest tag from being run, use this approach:

sbt> test-only -- -l DatabaseTest

(The second example uses a lowercase letter “L”.)

Notice that this uses a different version of the it method than was used in previous recipes. This example calls the two-argument version of the it method, whereas previous examples used the one-argument version. More on this shortly.

Discussion

Some software developers like to write long-running tests, such as database and web service tests, and others don’t. The ability to tag tests in this manner lets everyone have their way. Database and web service tests can be run when desired, and also excluded when desired. Just tag the database and web service tests with tags like DatabaseTest and WsTest, then include and exclude them when running your tests.

More generally, thinking beyond terms of long-running tests and short-running tests, this approach lets you test segments of your applications, in any way you want to define those segments. For instance, you can tag tests as Model, View, or Controller tests, or in any other way that makes sense for your application.

To add multiple tags to one BDD-style it test, add them as additional parameters:

it("Should start with no toppings", DatabaseTest, WsTest) {
  // test code here
}

To add tags to TDD-style tests, add the tags to your test method declarations:

test("new pizza has zero toppings", DatabaseTest, WsTest) {
  //assert(pizza.getToppings.size === 1)
  expectResult(0) {
    pizza.getToppings.size
  }
}

There are additional ways to tag tests. For instance, the ScalaTest documentation shows how to use taggedAs with an it test in a class that extends FlatSpec:

it must "subtract correctly" taggedAs(SlowTest, DbTest) in {
  val diff = 4 - 1
  assert(diff === 3)
}

See the links in the See Also section for more details.

To run tests that have both tags DatabaseTest and WsTest, separate their names by blank spaces inside quotes with the -n flag:

sbt> test-only -- -n "DatabaseTest WsTest"

Or to exclude those tests, do the same thing with the -l flag:

sbt> test-only -- -l "DatabaseTest WsTest"

For information on controlling which tags to include or exclude using other tools, such as running ScalaTest from the command line or Ant, see the ScalaTest filtering tests doc, and the Ant task filtering doc.

this post is sponsored by my books:
#1 New Release	FP Best Seller	Learn Scala 3	Learn FP Fast

ScalaTest 109: Mark your tests with tags to include or exclude them

Solution

Discussion

See Also

books by alvin