Cato's Place

During my recent look at the new JUnit 4.7 Core Rules I decided to identify the main differences between JUnit 3 and JUnit 4. In particular I wanted to look in to aspects that developers who came to JUnit at version 3 may not be aware of in JUnit 4

This curiosity was driven by a morbid fascination with change histories and the fact that on a recent project I noticed the lack of test suites. I had become accustomed to their use in JUnit 3 and a previous incarnation of the project had plenty of them. Having browsed some JUnit forums it became apparent to me that many developers had missed the i mprovedway to implement  Test Suites provided by JUnit 4.

Writing a JUnit 3 test suite meant creating a test class that extends from the base class for all JUnit 3 tests – TestCase. This class used a  static suite method to create a test suite. Each suite was added to the test suite using the addTestSuite() method that took the test suite class as its argument.

Heres how a JUnit 3 test suite looked for those (are there any) developers who never wrote one….

import junit.framework.Test;
import junit.framework.TestCase
import junit.framework.TestSuite

public class MyTestSuiteRunner extends TestCase {

  public static Test suite() {
    TestSuite suite = new TestSuite();
    suite.addTestSuite(ClazzToTestA.class);
    suite.addTestSuite(ClazzToTestB.class);
    return suite
  }
}

Compared to JUnit 3 test suites, the JUnit 4 version is much simpler to write and more concise. Using the Suite test runner in an @RunWith annotation the test suite classes can be added to the @SuiteClasses annotation.

import org.junit.runner.RunWith;
import org.junit.runners.Suite;
import org.junit.runners.Suite.SuiteClasses;

@RunWith(Suite.class)
@SuiteClasses( { ClazzToTestA.class, ClazzToTestB.class })
public class MyTestSuiteRunner {}

As can be seen the JUnit 4 version is much more concise and simple to write. So if you have made the change to JUnit 4 and are still using the old test suite creation methods now is a good time to upgrade to using code like this.

Continuing my look at the Core Rule implementations in JUnit 4.7 last week I had a look at the Error Collector Core Rule.

The Error Collector Core Rule enables tests to continue running even if assertions fail or errors occur. If is an implementation of the Verifier base class which I plan to look at next week. There are 3 methods in the ErrorCollector class that I found of interest:

• checkThat(final T value, Matcher<T> matcher)
• checkSucceeds(Callable<Object> callable)
• addError(Throwable error)

The first two methods deal with assertions and the last adds an error to the error list.

The checkThat(final T value, Matcher<T> matcher) method is used to test that an assertion is met. If the assertion fails then an assertion failure is added to the errors list and the test will fail. I created a test that used this method to check that the multiples of 2 in my test collection were all correct (full code can be found at the end of this article)

/*
 * Contains 2 failures both the matchers will report. Compare this to the
 * checkSucceeds example below.
 */
@Test
public void testMultiplesOf2() {
  int multiple = 0;
  for (int multipleOf2 : multiplesOf2) {
    // Will count the number of issues in this list
    // - 3*2 = 6 not 7, 5*2 = 10 not 11 : 2 Failures
    errorCollector.checkThat(2 * multiple, is(multipleOf2));
    multiple++;
  }
}

When the checkThat method is called the following Callback method is used.

checkSucceeds(new Callable<Object>() {
  public Object call() throws Exception {
    assertThat(value, matcher);
   return value;
  }
}

As can be seen the Callback method checks if the matcher matches the value. If it does not then a failure is added. The result in this test with the test data is 2 failures and a failed test.

In order to investigate the use of the checkSucceeds(Callable<Object> callable) method I created the following test method:

/*
 * Contains 2 failed assertions, but the checkSucceeds only register a single
 * failure per call, compared to checkThat which records each match failure
 */
@Test
public void testCallableMultiples() {
  errorCollector.checkSucceeds(new Callable<Object>() {
    public Object call() throws Exception {
      assertThat(2 * 2, is(5));
      assertThat(2 * 3, is(6));
      assertThat(2 * 4, is(8));
      assertThat(2 * 5, is(9));
      return null;
    }
  });
}

This test method displays how using this method enables the overriding of the checkThat methods Callback method. My test simply shows that only the first failure will be reported, not the 2 failures we may expect. This is probably not the best overriden Callback method, but it did show what I expected to. Would be interested to know of any real uses that this could be put to.

Lastly I looked at the addError method. My test method had 2 errors added using this method, and reports that 2 errors are created when run. This is similar to the examples within the JUnit release notes.

/*
 * Adds an error not a failure to the test. Will complete but with 2 errors.
 */
@Test
public void testAddingAnError() {
  assertThat(2 * 2, is(4));
  errorCollector.addError(new Throwable("Error Collector added an error"));
  assertThat(2 * 3, is(6));
  errorCollector.addError(new Throwable(
      "Error Collector added a second error"));
}

Why would you want to use this Core Rule? I found it hard to come up with a useful reason for doing so. If an assertion fails should the test complete? In which situations would this be useful – the parameterized tests within JUnit replicate this ability to run all tests even if a test fails. So, can this Core Rule be used to write custom parameterized test implementations? Be great to hear what uses there could be for this Core Rule.

The full Error Collector Core Rule test code is reproduced below:

import static org.hamcrest.CoreMatchers.is;
import static org.junit.Assert.assertThat;

import java.util.concurrent.Callable;

import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.ErrorCollector;

public class JUnitCoreErrorControllerRuleTest {

  private final int multiplesOf2[] = { 0, 2, 4, 7, 8, 11, 12 };

  /*
   * The ErrorCollector Rule allows execution of a test to continue after the
   * first problem is found and report them all at once
   */
  @Rule
  public ErrorCollector errorCollector = new ErrorCollector();

  /*
   * Contains 2 failures both the matchers will report. Compare this to the
   * checkSucceeds example below.
   */
  @Test
  public void testMultiplesOf2() {
    int multiple = 0;
    for (int multipleOf2 : multiplesOf2) {
      // Will count the number of issues in this list
      // - 3*2 = 6 not 7, 5*2 = 10 not 11 : 2 Failures
      errorCollector.checkThat(2 * multiple, is(multipleOf2));
      multiple++;
    }
  }

  /*
   * Contains 2 failed assertions, but the checkSucceeds only register a single
   * failure per call, compared to checkThat which records each match failure
   */
  @Test
  public void testCallableMultiples() {
    errorCollector.checkSucceeds(new Callable<Object>() {
      public Object call() throws Exception {
        assertThat(2 * 2, is(5));
        assertThat(2 * 3, is(6));
        assertThat(2 * 4, is(8));
        assertThat(2 * 5, is(9));
        return null;
      }
    });
  }

  /*
   * Adds an error not a failure to the test. Will complete but with 2 errors.
   */
  @Test
  public void testAddingAnError() {
    assertThat(2 * 2, is(4));
    errorCollector.addError(new Throwable(&quot;Error Collector added an error&quot;));
    assertThat(2 * 3, is(6));
    errorCollector.addError(new Throwable(
        &quot;Error Collector added a second error&quot;));
  }

}