Software Develoment Journey
Java, Spring Framework, How-to

Java test frameworks overview: Cucumber JVM, FEST 2.x, catch-exception

Beauty and curse same time of Java ecosystem is the variety of available frameworks and solutions. There are plenty of ready to use solutions for most of problems that we meet on our daily basis and the tricky part is only to choose the right one. It also applies to testing frameworks. Times when the only tool you use is JUnit are hopefully gone – now there are plenty of very high quality frameworks that help to write any kind of automated tests not only faster but what is more important tests reliability and maintainability factor increases a lot.

Today I would like to share with you 3 testing frameworks/tools that I find very useful and handy.

Cucumber JVM

Behavior Driven Development becomes more and more popular. In my opinion it is a great way not only writing acceptance tests but developing software in general. There are couple of BDD frameworks for Java and since march 2012 we can find real Java port of one of the most important Ruby frameworks – CucumberCucumber JVM.

Cucumber supports Gherkin language to define steps which makes it easy and natural to create specification. Another advantage of Gherkin is that there are cucumber-like frameworks for many languages and platforms which makes it grate to use in organization when team members need to learn only one way of defining steps.

Complete example of usage Cucumber-JVM can be found in version 1.0.0 official announcement. As you can see there – it does not look hard. Hard part begins when it comes to write testable step definitions – it is something for another post. That’s how example feature can look like:

1
2
3
4
5
6
7
Feature: As a player I want to be able to send mail invitation to my friends

Scenario: player sends successful invitation
        Given player "John Doe"
        When "John Doe" invites "jane.smith@somemail.com"
        Then 1 mail is sent
        And invitation from "John Doe" to "jane.smith@someemail.com" is saved

And implementation of those steps: (just an example)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
public class InvitationSteps {
    @Autowired
    private PlayerRepository playerRepository;

    @Autowired
    private InvitationRepository invitationRepository;

    @Autowired
    private SmtpServer mailServer;

    @Given("^player \"([^\"]*)\"$")
    public void player(String name) throws Throwable {
        playerRepository.save(new Player(name));
    }

    @When("^\"([^\"]*)\" invites \"([^\"]*)\"$")
    public void invites(String invitingPlayerName, String invitedMail) throws Throwable {
        Player player = playerRepository.findByName(invitingPlayerName);
        player.invite(invitedMail);
    }

    @Then("^(\\d+) mail is sent$")
    public void mail_is_sent(int count) throws Throwable {
        assertThat(mailServer.getEmailCount()).isEqualTo(count);
    }

    @Then("^invitation from \"([^\"]*)\" to \"([^\"]*)\" is saved$")
    public void invitation_from_to_is_saved(String from, String to) throws Throwable {
        List<Invitation> invitationList = invitationRepository.findInvitations(from, to);

        assertThat(invitationList).hasSize(1);
    }
}

Steps are reusable across whole tested applications. Thanks to that sometimes we don’t need to write single line of code to test another scenario!

Cucumber-JVM even when is quite young provides mature feature base: integrates transparently with Spring Framework, generates HTML and JSON reports, there is a Jenkins plugin for that as well.

I started to use Cucumber-JVM before version 1.0 was released. Now I am on second project where integration tests are defined in Cucumber features and I am not planning to turn back.

FEST Assertions 2.x

I wrote about FEST in March this year. FEST is a direct replacement for JUnit and Hamcrest assertions. I am happy to say that this year Alex Ruiz is back on this project which evolved from version 1.4 to 2.0M8. Alex describes version 2.x as:

This is an attempt to improve our codebase from the previous 1.x releases in terms of extensibility, code reuse, and code organization.

In version 2.0 we find new assertions for standard classes and also simplified way of creating custom assertions. New assertions for standard classes is something that IDE will help you to find out. What I find very interesting in FEST is creating custom assertions. Let’s assume we have a class Player:

1
2
3
4
public class Player {
    private String name;
    private int coins;
}

Classic FEST approach gives us fluent interface to test player’s properties:

1
2
assertThat(player.getName()).isEqualTo("alex");
assertThat(player.getCoins()).isPositive();

That’s acceptable but would look much more readable if we could just say:

1
assertThat(player).hasName("alex").hasCoins();

In order to achieve that we need to write custom class containing assertions for Player class:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
public class PlayerAssert extends AbstractAssert<PlayerAssert, Player> {
    public PlayerAssert(Player actual) {
        super(actual, PlayerAssert.class);
    }

    public PlayerAssert hasName(String name) {
        isNotNull();

        Assertions.assertThat(actual.getName())
                .overridingErrorMessage("Expected player's name to be <%s> but was <%s>", name, actual.getName())
                .isEqualTo(name);

        return this;
    }

    public PlayerAssert hasCoins() {
        isNotNull();

        Assertions.assertThat(actual.getCoins())
                .overridingErrorMessage("Expected positive amount of coins but was <%d>", actual.getCoins())
                .isPositive();

        return this;
    }
}

The only thing that is left is to create single entry for all your assertions. Whole process is described in FEST Wiki.

I agree that result of that is handy but writing asserts for each and every tested classes means quite big effort. And fortunately this problem is also solved (or actually being solved since there is no final solution yet). Joel Costigliola (“aka FEST Hero) works on great project FEST Assertion Generator – tool used to generate assertions for single classes or all classes in packages. For now there is a command line interface and Eclipse plugin which definitely limits its usage but I am hoping that Maven plugin will be released soon.

catch-exception

Handling exceptions in unit tests never felt really naturaly. In JUnit there are 2 basic options to do it:

try, fail, catch approach

1
2
3
4
5
6
try {
    someService.foo();
    failBecauseExceptionWasNotThrown(IllegalArgumentException.class);
} catch (IllegalArgumentException e) {
    assertThat(e).hasMessage("some message");
}

That feels clumsy.

@Test(expected = …)

1
2
3
4
@Test(expected = IllegalArgumentException.class)
public void testFoo() {
    someService.foo();
}

That looks more readable but takes out possibility to make any additional assertion to thrown exception.

There is better way to do it. Blazingly simple library catch-exception makes testing thrown exceptions something natural and readable.

catch-exception approach

The methods of CatchException class catch and verify exceptions in a single line of code and make them available for further analysis.

1
2
3
4
5
6
7
8
9
import static com.googlecode.catchexception.CatchException.catchException;
import static com.googlecode.catchexception.CatchException.caughtException;

(...)
@Test
public void testFoo() {
    catchException(someService).foo(request);
    assertThat(caughtException()).isInstanceOf(IllegalArgumentException.class).hasMessage("some message");
}

There is not much to add here. Library does what is was ment to do. Simple one-liners handles exceptions in the most readable and flexible way.

Summary

When Cucumber is a really BIG thing, FEST 2.x and catch-exception are small libraries that make life easier. There are much more tools like that. The most heavily used framework by me is Mockito. I can’t really imagine unit tests without it.

What libraries can be found in your <scope>test</test>?

If you enjoyed this post, then make sure you subscribe to my RSS feed