4.21 Repeatedly Testing the Same Method

4.21.1 Problem

You want to test a method with a wide range of input data. You are not sure if you should write a different test for each combination of input data, or one huge test that checks every possible combination.

4.21.2 Solution

Write a suite( ) method that iterates through all of your input data, creating a unique instance of your test case for each unique input. The data is passed to the test cases through the constructor, which stores the data in instance fields so it is available to the test methods.

4.21.3 Discussion

You often want to test some piece of functionality with many different combinations of input data. Your first impulse might be to write a different test method for each possible combination of data; however, this is tedious and results in a lot of mundane coding. A second option is to write a single, big test method that checks every possible combination of input data. For example:

public void testSomething(  ) {
    Foo foo = new Foo(  );
    // test every possible combination of input data
    assertTrue(foo.doSomething(false, false, false);
    assertFalse(foo.doSomething(false, false, true);
    assertFalse(foo.doSomething(false, true, false);
    assertTrue(foo.doSomething(false, true, true);
    ...etc
}

This approach suffers from a fatal flaw. The problem is that the test stops executing as soon as the first assertion fails, so you won't see all of the errors at once. Ideally, you want to easily set up a large number of test cases and run them all as independent tests. One failure should not prevent the remaining tests from running.

To illustrate this technique, Example 4-16 contains a utility for determining the background color of a component. The getFieldBackground( ) method calculates a different background color based on numerous parametersfor example, whether a record is available and whether the current data is valid.

Example 4-16. UtilComponent

public class UtilComponent {
    public static final int ADD_MODE = 1;
    public static final int RECORD_AVAILABLE_MODE = 2;
    public static final int NO_RECORD_AVAILABLE_MODE = 3;

    public static final int KEY_FIELD = 100;
    public static final int REQUIRED_FIELD = 101;
    public static final int READONLY_FIELD = 102;
    public static final int NORMAL_FIELD = 103;

    public static final Color FIELD_NORMAL_BACKGROUND = Color.WHITE;
    public static final Color FIELD_ERROR_BACKGROUND = Color.PINK;
    public static final Color FIELD_REQUIRED_BACKGROUND = Color.CYAN;
    public static final Color FIELD_DISABLED_BACKGROUND = Color.GRAY;

    public static Color getFieldBackground(
            int screenMode,
            int fieldType,
            boolean valid,
            boolean requiredConditionMet) {
        if (fieldType == READONLY_FIELD
                || screenMode == NO_RECORD_AVAILABLE_MODE
                || (fieldType == KEY_FIELD && screenMode != ADD_MODE)) {
            return FIELD_DISABLED_BACKGROUND;
        }

        if (!valid) {
            return FIELD_ERROR_BACKGROUND;
        }
        if ((fieldType == KEY_FIELD || fieldType == REQUIRED_FIELD)
                && !requiredConditionMet) {
            return FIELD_REQUIRED_BACKGROUND;
        }
        return FIELD_NORMAL_BACKGROUND;
    }

    public static String colorToString(Color color) {
        if (color == null) {
            return "null";
        }
        if (color.equals(FIELD_DISABLED_BACKGROUND)) {
            return "FIELD_DISABLED_BACKGROUND";
        }
        if (color.equals(FIELD_ERROR_BACKGROUND)) {
            return "FIELD_ERROR_BACKGROUND";
        }
        if (color.equals(FIELD_REQUIRED_BACKGROUND)) {
            return "FIELD_REQUIRED_BACKGROUND";
        }
        if (color.equals(FIELD_NORMAL_BACKGROUND)) {
            return "FIELD_NORMAL_BACKGROUND";
        }
        return color.toString(  );
    }
}

There are 48 possible combinations of inputs to the getFieldBackground( ) method, and 4 possible return values. The test case defines a helper class that encapsulates one combination of inputs along with an expected result. It then builds an array of 48 instances of this class, 1 per combination of input data. Example 4-17 shows this portion of our test.

Example 4-17. Defining the test data

public class TestUtilComponent extends TestCase {

    private int testNumber;

    static class TestData {
        int screenMode;
        int fieldType;
        boolean valid;
        boolean requiredConditionMet;
        Color expectedColor;

        public TestData(int screenMode, int fieldType, boolean valid,
                        boolean requiredConditionMet, Color expectedColor) {
            this.screenMode = screenMode;
            this.fieldType = fieldType;
            this.valid = valid;
            this.requiredConditionMet = requiredConditionMet;
            this.expectedColor = expectedColor;
        }
    }

    private static final TestData[] TESTS = new TestData[] {
        new TestData(UtilComponent.ADD_MODE, // 0
                     UtilComponent.KEY_FIELD,
                     false, false,
                     UtilComponent.FIELD_ERROR_BACKGROUND),
        new TestData(UtilComponent.ADD_MODE, // 1
                     UtilComponent.KEY_FIELD,
                     false, true,
                     UtilComponent.FIELD_ERROR_BACKGROUND),
        new TestData(UtilComponent.ADD_MODE, // 2
                     UtilComponent.KEY_FIELD,
                     true, false,
                     UtilComponent.FIELD_REQUIRED_BACKGROUND),

        ...continue defining TestData for every possible input

The test extends from the normal JUnit TestCase base class, and defines a single private field called testNumber. This field keeps track of which instance of TestData to test. Remember that for each unit test, a new instance of TestUtilComponent is created. Thus, each instance has its own copy of the testNumber field, which contains an index into the TESTS array.

The TESTS array contains every possible combination of TestData. As you can see, we include comments containing the index in the array:

new TestData(UtilComponent.ADD_MODE, // 0

This index allows us to track down which test cases are not working when we encounter failures. Example 4-18 shows the remainder of our test case, illustrating how the tests are executed.

Example 4-18. Remainder of TestUtilComponent

    public TestUtilComponent(String testMethodName, int testNumber) {
        super(testMethodName);
        this.testNumber = testNumber;
    }

    public void testFieldBackgroundColor(  ) {
        TestData td = TESTS[this.testNumber];

        Color actualColor = UtilComponent.getFieldBackground(td.screenMode,
                td.fieldType, td.valid, td.requiredConditionMet);

        assertEquals("Test number " + this.testNumber + ": ",
                UtilComponent.colorToString(td.expectedColor),
                UtilComponent.colorToString(actualColor));
    }

    public static Test suite(  ) {
        TestSuite suite = new TestSuite(  );
        for (int i=0; i<TESTS.length; i++) {
            suite.addTest(new TestUtilComponent("testFieldBackgroundColor", i));
        }
        return suite;
    }
}

Our constructor does not follow the usual JUnit pattern. In addition to the test method name, we accept the test number. This is assigned to the testNumber field, and indicates which data to test.

The testFieldBackgroundColor( ) method is our actual unit test. It uses the correct TestData object to run UtilComponent.getFieldBackground( ), using assertEquals( ) to check the color. We also use UtilComponent to convert the color to a text string before doing the comparison. Although this is not required, it results in much more readable error messages when the test fails.

The final portion of our test is the suite( ) method. JUnit uses reflection to search for this method. If found, JUnit runs the tests returned from suite( ) rather than using reflection to locate testXXX( ) methods. In our case, we loop through our array of test data, creating a new instance of TestUtilComponent for each entry. Each test instance has a different test number, and is added to the TestSuite. This is how we create 48 different tests from our array of test data.

Although we have a hardcoded array of test data, there are other instances where you want to make your tests more customizable. In those cases, you should use the same technique outlined in this recipe. Instead of hardcoding the test data, however, you can put your test data into an XML file. Your suite( ) method would parse the XML file and then create a TestSuite containing the test data defined in your XML.