How to Name Tests for Maintainability

A Suggestion for More Conciseness

Naming tests has little impact. The tests fail or succeed no matter what they are named. But names matter and as a user and author of a test suite, I have several requirements:

1. In order to avoid duplicates, test case names should describe the test’s content in a schematic way.
2. In order to be able to understand the meaning of a failing test and find the cause fast, the test case names should describe the test’s content sufficiently.
3. In order to avoid confusion and wrong assumptions, test case names must never lie about the test’s content.

And there may be many more.

Existing Suggestions

Of course there are other suggestions out there. In his article “7 Popular Unit Test Naming Conventions” Ajitesh Kumar gives a good overview.

I think most suggestions I read or heard about so far, sort in following three categories:

Minimalism

Some developers argue to number test cases instead of naming them and concentrate on making the code itself readable.

<UnitOfWork>Test.test<Counter>

This approach clearly makes sure that test names never lie about the test content since they don’t say anything about it. I will be forced to read the code to understand a test failure, which probably takes longer than just reading the test case name. Also those names won’t help to avoid duplicates.

The idea might seem a bit extreme, but it makes writing short and well readable tests extremely important, which might be much more important than thinking too much about naming them.

Comprehensiveness

Others suggest a rather strict and comprehensive formal naming. The most popular articles seems to be the “Naming standards for unit tests” by Roy Osherove, which suggest the scheme

<UnitOfWork>_<StateUnderTest>_<ExpectedBehavior>

With this approach test failures are pretty easy to understand since the test case name contains basically everything the test does and expects. It might also do a good job on avoiding duplicate tests as long as the authors have a common understanding on how to describe the <StateUnderTest> and the <ExpectedBehavior>.

However, these test names can get long and confusing. Developers are likely to copy paste around and only update the <StateUnderTest>, forgetting to change the <ExpectedBehavior> part. Even when reviewed this might slip, as the reviewers are likely to focus on the code changes. Also this –and similar naming schemes– basically double the developers’ work. They need to write the test once in code and once in English.

In the end the problem might be a test suite lying about the content of the tests.

Informalism

The approach I find most in reality is the informal one. In that case basically there is no explicit rule on how to name tests. This obviously leads to a mixture of naming and all the bad effects of both other extremes are likely to apply here too.

test<UnitOfWork>Should<ExpectedBehavior>With<StateUnderTest>

<UnitOfWork>Does<ExpectedBehavior>If<StateUnderTest>

itShould<ExpectedBehavior>

My Suggestion: Conciseness

In addition to the above reasons, I really don’t think that test case names are a one-size-fits-all thing. I think each test type might deserve a slightly different naming scheme.

Apart from this, my idea is inspired by Chris Beams’ Article “How to Write a Git Commit Message”, taking the test case’s name as a subject line and test code as the body.

Here are my rules for naming test cases:

Rule 1: Don’t repeat the context: If something is obvious from context, don’t put it in the test case name. Especially the subject of a test case should be obvious from context.

Rule 2: Start with the most useful: Don’t hide the most useful information at the end of the test case name. Put it in front!

Rule 3: Avoid filler words: Words like “should” usually don’t carry any information. Rather use simple third person language. E.g. instead of “it should return a valid object”, use “it returns valid object”.

Rule 4: Don’t specify the happy case: Often it is rather obvious how something should be used in order to work correctly. So, we should leave that implicit, while in cases where we deviate from the happy case make that deviation explicit.

Rule 5: Don’t state expectations: Stating what outcome we expect from a test case is quite tempting. However, the actual assertions in the test already do that. By putting expectations in the test case name we duplicate that information. There’s a high chance that the expectation becomes outdated over time, and is more or less specific than what the test actually tests. Just leave the specific expectations to the test code.

Unit & Integration Tests

Unit tests are the most technical ones. Their names should be very close to the code. E.g. by naming classes, methods and functions explicitly.

The test class or file should therefore be named and placed close to the class or code under test. In Java it is a common habit to place the test class in the same package and name it after the class under test, which makes absolute sense.

Let’s take a (bad) example in JUnit 5:

class BasketAdapterTest {

    @DisplayName("given a valid customerId and a machine " +
        "token basketAdapter.getBasket should return a " +
        "BasketResultDto")
    void testGetBasket1() {
        // ...
    }
}

Applying rule 1: Don’t repeat the context: Since the class under test is in the test class name, we don’t need it in the case’s name. The method which is called however makes sense to narrow down the text’s context.

BasketAdapterTest.given a valid customerUuid and a machine token getBasket should return a BasketResultDto

Applying rule 2: Start with the most useful: The method being tested is the most useful information in Unit Tests, as it helps most to find the cause. By making the case’s name start with it, the cases are naturally grouped.

BasketAdapterTest.getBasket should return a BasketResultDto given a valid customerUuid and a machine token

Applying rule 3: Avoid filler words: should return can be shortened to returns and the as don’t carry much information either.

BasketAdapterTest.getBasket returns BasketResultDto for valid customerUuid and machine token

Applying rule 4: Don’t specify the happy case: I think it is rather obvious that we use valid objects for mandatory parameters. So let’s omit valid customerUuid and machine token.

BasketAdapterTest.getBasket returns BasketResultDto

That way a deviating test stands out much more with the appended explicit deviation from the happy case.

BasketAdapterTest.getBasket throws IllegalArgumentException for invalid machine token

Rule 5: Don’t state expectations: If the test fails, the assertion output should tell you what was expected and how it differs from the actual behaviour.

BasketAdapterTest.getBasket

BasketAdapterTest.getBasket invalid machine token

So we end with a simple pattern for unit test names:

<classUnderTest>Test.<methodUnderTest> [<happyCaseDiffStateUnderTest>]

Let’s check the user experience of this naming scheme:

1. I need to change the method calculateTotalPrice of the Basket class. So I use Ctrl+N/⌘+O and type “BasketTest” to find the right test class. Then I look for all test cases starting with “calculateTotalPrice” and check for tests on behavior I’d like to change.
2. A test report says BasketTest.calculateTotalPrice is failing at assertThat(basket.calculateTotalPrice).isEqualTo(articlePrice + shippingFee). So I check the calculateTotalPrice method in Basket and look for something like “shippingFee”.
3. I need to change the test and the implementation of calculateTotalPrice: The shipping fee is no longer added if the sum of the articles prices is above 100€. So I need to change the content of the basket in BasketTest.calculateTotalPrice to below 100€ and add one new test BasketTest.calculateTotalPrice articles ≥ 100€. Since the happy case is not specified in the test case name, I didn’t need to update it.

Parameterized Unit Tests

Parametrized or data-driven tests (see JUnit 5 documentation) are a great way to test a lot of different inputs with the same test code. Concise naming also works for them:

class BasketAdapterTest {

    @ParameterizedTest(name = "addQuanitity({0})")
    @ValueSource(ints = {1, 2, 5, 9})
    void testAddQuantity1(int quanitity) {
        // ...
    }
}

API Tests

API tests are a special thing. On the one hand these tests might seem quite technical as well, but on the other hand they are –if well written– and exact executable documentation of a service’s API.

The test class or file should be named after the most common name of the API under test. For RESTful APIs this is usually the resource’s name.

Here is a bad example for an API test name:

class BasketApiTest {

    @DisplayName("The Basket-API should return 200 and a " +
        "Basket JSON on GET for a valid basket UUID")
    void testGetBasket1() {
        // ...
    }
}

Applying rule 1: Don’t repeat the context. The test class name in this case is already called “BasketApiTest”, so we can remove the piece of information.

BasketApiTest.should return 200 and a Basket JSON on GET for a valid basket UUID

Applying rule 2: Start with the most useful: To have a formal standard on naming API tests, let’s put the method — in this case GET — in front followed by the path. For many tests the path will be the same in the whole test class, but it might contain variables and it makes manual verification (e.g. via curl) pretty easy.

BasketApiTest.GET /api/basket/:id should return 200 and a Basket JSON for a valid basket UUID

Applying rule 3: Avoid filler words: The name can be a bit more concise by replacing the should return with returns.

BasketApiTest.GET /api/basket/:id returns 200 and a Basket JSON for a valid basket UUID

Applying rule 4: Don’t specify the happy case: I think providing a valid basket UUID can be assumed.

BasketApiTest.GET /api/basket/:id returns 200 and a Basket JSON

Providing an invalid ID is interesting, though.

BasketApiTest.GET /api/basket/:id return 404 and error JSON for invalid id

Applying rule 5: Don’t state expectations: Let’s rely on assertion output.

BasketApiTest.GET /api/basket/:id

BasketApiTest.GET /api/basket/:id for invalid id

Generally we get to the pattern

<apiUnderTest>ApiTest.<method> <path> [<happyCaseDiffStateUnderTest>]

Let’s check our use cases:

1. The basket JSON object returned by the Basket API should be extended. So I Ctrl+N/⌘+O and type “BasketApiTest”. So I look for tests starting with GET /api/basket/ and update the assertions on the returned object.
2. I find that BasketApi.GET /api/basket/:id fails at assertThat(response.code()).isEqualTo(200). So I look into the BasketApiController and drill down to find the place where the response code is determined.
3. Changing the basket JSON in any way does not require me to change any test case name for it is only returned in the happy case.

GUI Tests

There are different types of GUI tests. Some are actually system, use case or end-to-end tests, which happen to use the GUI as the outermost interface for the test. Those are discussed below. This part is about tests, which test features of the UI itself. So clearly the unit under test is the GUI or a single element of the UI. E.g. a smart button, a form, or a menu.

class BasketButtonGuiTest {

    @DisplayName("The basket button should turn into a quantity field " +
        "when it is clicked once and is activated")
    void testGetBasket1() {
        // ...
    }
}

Applying rule 1: Don’t repeat the context: The test’s name is BasketButtonGuiTest, so all the cases will be about that button. It is probably a good idea to narrow the context of each test to as few GUI elements as possible.

BasketButtonGuiTest.it should turn into a quantity field when it is clicked once and is activated

Applying rule 2: Start with the most useful: In this case, the most significant important thing is probably the interaction happening to the GUI element. In this case a click.

BasketButtonGuiTest.clicked once it should turn into a quantity field when it is activated

Applying rule 3: Avoid filler words: should turn can be simplified to turns, and the a can be omitted. Even clicked once can be shortened to click once. The latter might be a matter of taste, though.

BasketButtonGuiTest.click once turns into quantity field when activated

Applying rule 4: Don’t specify the happy case: I think it is fair to assume that the button is activated.

BasketButtonGuiTest.click once turns into quantity field

An example a deviating test case:

BasketButtonGuiTest.click once it shows error message deactivated

Applying rule 5: Don’t state expectations: I admit, that this rule may seem even more extreme applied to GUIs as the expected behaviour might be quite complex. However, when we change that behaviour, we will need to change all the test names with it and the test code should be sufficient to tell us about the details.

BasketButtonGuiTest.click once

BasketButtonGuiTest.click once deactivated

Generally we get to the pattern

<guiUnderTest>GuiTest.<interaction> [<happyCaseDiffStateUnderTest>]

Conclusion

I personally like naming tests for conciseness for a couple of reasons:

• I have a schematic way of naming things. My inner monk is happy with it.
• Nothing in the tests’ names is boring. Their information density is high and I can quickly and clearly see how the test intention of one test is different from another.
• The focus on what is being tested forces me to write concise test code, choosing good variables, and expressive assertion code.
• The probability for lying or misleading test names is close to 0 with this naming scheme.

Note that the rules are numbered. If you think the fully concise naming scheme is too drastic, feel free to skip the rules you don’t like!