Brian CoynerExcellence In Software Engineering

OCUnit Parameterized Test Case

August 30, 2011

This article shows how to implement a "Parameterized" (a.k.a data-driven) test case in the OCUnit test framework. The example described in this article is similar to the technique used in the Java Extreme Programming Cookbook, which I co-authored back in 2002/2003. I use the techniques described in this article as much as I can when writing unit tests.

For readers familiar with JUnit, this is similar to providing an implementation of the public static Test suite() method.

JUnit 4+ provides direct support for building "Parameterized" tests using the @RunWith(Parameterized.class) annotation.

A parameterized test case is a standard test case (i.e. a subclass of SenTestCase), usually containing a single test method that executes multiple times with a different set of input expected values. The goal is to "add test data" to our test suite, not "add test methods".

There are four core steps to building an OCUnit parameterized test case:

  1. Create a new subclass of SenTestCase (same as any other OCUnit test case)
  2. Override the class method defaultTestSuite to dynamically generate a new SenTestSuite
  3. Add properties/ ivars to the test case that represent the input value(s) and expected output value(s)
  4. Create one or more standard test methods that exercise the input/ expected values.

What Are We Going To Test?

This article uses a simple class named BTCPreferredOrderIndexGenerator. The "generator" takes an NSDate and converts the year, month and day components into an unsigned long long (64-bit value). The "generator" also pads the value with ten zeros. For example, the "generator" converts January 9, 2010 to 201001010000000000.

Production Code Implementation

Here is the complete production code we are going to test.

BTCPreferredOrderIndexGenerator.h

#import <Foundation/Foundation.h>

extern NSString * const kBTCInvalidDate;

@interface BTCPreferredOrderIndexGenerator : NSObject
- (unsigned long long)generateFromDate:(NSDate *)date;
@end

BTCPreferredOrderIndexGenerator.m

#import "BTCPreferredOrderIndexGenerator.h"

NSString * const kBTCInvalidDate = @"BTCInvalidDate";

@implementation BTCPreferredOrderIndexGenerator

- (unsigned long long)generateFromDate:(NSDate *)date
{
    if (date == nil) {
        @throw [NSException exceptionWithName:kBTCInvalidDate reason:@"date is nil" userInfo:nil];
    }

    NSCalendar *calendar = [NSCalendar currentCalendar];
    NSUInteger dateComponents = (NSYearCalendarUnit | NSMonthCalendarUnit | NSDayCalendarUnit);
    NSDateComponents *components = [calendar components:dateComponents fromDate:date];

    NSInteger year = [components year];
    NSInteger month = [components month];
    NSInteger day = [components day];

    return ((year * 10000) + (month * 100) + (day)) * 10000000000;
}

@end

Creating a Parameterized Tests

OCUnit does not appear to provide direct support for "Parameterized" tests. However, OCUnit does allow for manually creating test suites by overriding the defaultTestSuite class method (see Step 3 below). The test suite is the cross-product of test methods and test data.

A single test execution represents a unique instance of a test case class (instance and test method). This means that we can add ivars to our test case whose values are unique to each test method execution. The properties/ ivars are the "generator's" input and expected values.

Step 1: Declare the test case public interface

@interface BTCPreferredOrderIndexGeneratorTest : SenTestCase

@property (nonatomic, assign) BTCPreferredOrderIndexGenerator *generator;

// These are the instance variables used by a single test method execution
@property (nonatomic, assign) unsigned long long expectedValue;
@property (nonatomic, copy) NSDate *date;
@property (nonatomic, retain) NSException *expectedException;

// This is a helper method to dynamically build a "Parameterized" test for every "test" method found in this test case with a
// single set of input data.
+ (void)addTestWithExpectedValue:(unsigned long long)expectedValue
                            date:(NSDate *)date
               expectedException:(NSException *)expectedExceptionOrNil
                     toTestSuite:(SenTestSuite *)testSuite;
@end
Our test case class now contains three properties:
  • expectedValue - the value generated by the BTCPreferredOrderIndexGenerator. Used only if the expectedException is not nil.
  • date - the input date used by the BTCPreferredOrderIndexGenerator to generate the index number.
  • expectedException - optional value used to inform the test that BTCPreferredOrderIndexGenerator should throw an exception.
@implementation BTCPreferredOrderIndexGeneratorTest

@synthesize generator = _generator;

@synthesize expectedValue = _expectedValue;
@synthesize date = _date;
@synthesize expectedException = _expectedException;

// Custom initializer method used for each instance of a test execution. The NSInvocation represents a
// unique test instance + test method.
- (id)initWithInvocation:(NSInvocation *)anInvocation
           expectedValue:(unsigned long long)value
                    date:(NSDate *)inputDate
       expectedException:(NSException *)expectedExceptionOrNil
{

    // NOTE: Call the SenTestCase designated initializer with the given invocation
    self = [super initWithInvocation:anInvocation];
    if (self) {
        [self setExpectedValue:value];
        [self setDate:inputDate];
        [self setExpectedException:expectedExceptionOrNil];
    }

    return self;
}

- (void)setUp
{
    _generator = [[BTCPreferredOrderIndexGenerator alloc] init];
}

- (void)tearDown
{
    [self setGenerator:nil];
    [self setDate:nil];
    [self setExpectedException:nil];
}

Step 2: Add a Test Method

- (void)testGenerateFromDate
{
    if ([self expectedException] == nil) {
        STAssertEquals(_expectedValue, [_generator generateFromDate:[self date]], @"Generated Preferred Order Index");
    }
    else {
        @try {
            [_generator generateFromDate:_date];
            STFail(@"Generator should raise exception.");
        }
        @catch (NSException *exception) {
            // You should test the exception however it makes sense in your application...
            STAssertEqualObjects([_expectedException class], [exception class], @"Exception Class.");
            STAssertEqualObjects([_expectedException name], [exception name], @"Exception Name.");
            STAssertEqualObjects([_expectedException reason], [exception reason], @"Exception reason.");
        }  
    }
}

Step 3: Dynamically Generate the Test Suite

Every SenTestCase has a class method named defaultTestSuite, that by default, dynamically scans your test case for instance methods that return void, start with the name "test" and take zero arguments. To make a "Parameterized" test we need to override the defaultTestSuite class method and manually build a SenTestSuite.

#pragma mark Class Methods To Generate The Test Suite

+ (id)defaultTestSuite
{
    // Create a new SenTestSuite using the name of our class.
    SenTestSuite *testSuite = [[SenTestSuite alloc] initWithName:NSStringFromClass(self)];

    // Add "Parameterized" data.
    //
    // Each call to "addTestWithExpectedValue" adds a new test to the test suite with the given values.
    //
    // The use of the "helper" method allows the programmer to easily scan the set of test data.
    [self addTestWithExpectedValue:201001090000000000 date:_date(2010, 1, 9) expectedException:nil toTestSuite:testSuite];
    [self addTestWithExpectedValue:201001100000000000 date:_date(2010, 1, 10) expectedException:nil toTestSuite:testSuite];
    [self addTestWithExpectedValue:201001110000000000 date:_date(2010, 1, 11) expectedException:nil toTestSuite:testSuite];
    [self addTestWithExpectedValue:201010090000000000 date:_date(2010, 10, 9) expectedException:nil toTestSuite:testSuite];
    [self addTestWithExpectedValue:201010100000000000 date:_date(2010, 10, 10) expectedException:nil toTestSuite:testSuite];
    [self addTestWithExpectedValue:201010110000000000 date:_date(2010, 10, 11) expectedException:nil toTestSuite:testSuite];
    [self addTestWithExpectedValue:999921310000000000 date:_date(2010, 10, 11) expectedException:nil toTestSuite:testSuite];


    { // nil date should throw an exception
        NSException *expectedException = [[NSException alloc] initWithName:BTCInvalidDate reason:@"date is nil" userInfo:nil];
        [self addTestWithExpectedValue:-1 date:nil expectedException:expectedException toTestSuite:testSuite];
        [expectedException release];
    }

    return [testSuite autorelease];
}

+ (void)addTestWithExpectedValue:(unsigned long long)expectedValue
                            date:(NSDate *)date
               expectedException:(NSException *)expectedException
                     toTestSuite:(SenTestSuite *)testSuite
{
    // NOTE: `testInvocations` is implemented by the SenTestKit framework, which scans our test case class for "test" methods.
    //       Each call to [self testInvocations] returns an array of unique NSInvocation instances. This is the exact behavior
    //       we need to correctly build a cross-product of test data to test methods that make up our test suite.
    //
    //       The array count is equal to the number of "test" methods found in the test case class.
    //       For example, if there are three test methods, then the returned array contains three unique `NSInvocation` objects.
    NSArray *testInvocations = [self testInvocations];

    // This array of NSInvocation objects contains a single object that points to our single test method.
    for (NSInvocation *testInvocation in testInvocations) {

        // Create a new instance of our test case for each method found using the given set of parameters.
        SenTestCase *test = [[BTCPreferredOrderIndexGeneratorTest alloc]
                initWithInvocation:testInvocation
                     expectedValue:expectedValue
                              date:date
                 expectedException:expectedException];

        // Add the new test instance to the suite. The OCUnit framework eventually executes the entire test suite.
        [testSuite addTest:test];

        [test release];
    }
}

Our test case executes just like any other test case in your project. After executing the tests you will see the following output.

Test Suite 'BTCPreferredOrderIndexGeneratorTest' started at ... .
Test Case '-[BTCPreferredOrderIndexGeneratorTest testGenerateFromDate]' started.
Test Case '-[BTCPreferredOrderIndexGeneratorTest testGenerateFromDate]' passed (0.000 seconds).
Test Case '-[BTCPreferredOrderIndexGeneratorTest testGenerateFromDate]' started.
Test Case '-[BTCPreferredOrderIndexGeneratorTest testGenerateFromDate]' passed (0.000 seconds).
Test Case '-[BTCPreferredOrderIndexGeneratorTest testGenerateFromDate]' started.
Test Case '-[BTCPreferredOrderIndexGeneratorTest testGenerateFromDate]' passed (0.000 seconds).
Test Case '-[BTCPreferredOrderIndexGeneratorTest testGenerateFromDate]' started.
Test Case '-[BTCPreferredOrderIndexGeneratorTest testGenerateFromDate]' passed (0.000 seconds).
Test Case '-[BTCPreferredOrderIndexGeneratorTest testGenerateFromDate]' started.
Test Case '-[BTCPreferredOrderIndexGeneratorTest testGenerateFromDate]' passed (0.000 seconds).
Test Case '-[BTCPreferredOrderIndexGeneratorTest testGenerateFromDate]' started.
Test Case '-[BTCPreferredOrderIndexGeneratorTest testGenerateFromDate]' passed (0.000 seconds).
Test Case '-[BTCPreferredOrderIndexGeneratorTest testGenerateFromDate]' started.
Test Case '-[BTCPreferredOrderIndexGeneratorTest testGenerateFromDate]' passed (0.000 seconds).
Test Case '-[BTCPreferredOrderIndexGeneratorTest testGenerateFromDate]' started.
Test Case '-[BTCPreferredOrderIndexGeneratorTest testGenerateFromDate]' passed (0.000 seconds).
Test Suite 'BTCPreferredOrderIndexGeneratorTest' finished at ... .
Executed 8 tests, with 0 failures (0 unexpected) in 0.001 (0.003) seconds

Full Test Case Implementation

For convenience, here is the entire test case implementation.

#import "BTCPreferredOrderIndexGenerator.h"

#pragma mark Test Case Interface

@interface BTCPreferredOrderIndexGeneratorParameterizedTest : SenTestCase

@property (nonatomic, assign) BTCPreferredOrderIndexGenerator *generator;

@property (nonatomic, assign) unsigned long long expectedValue;
@property (nonatomic, copy) NSDate *date;
@property (nonatomic, retain) NSException *expectedException;

// Method to dynamically build a "Parameterized" test for every "test" method found in this test case with a single set of
// input data
+ (void)addTestWithExpectedValue:(unsigned long long)expectedValue
                            date:(NSDate *)date
               expectedException:(NSException *)expectedException
                     toTestSuite:(SenTestSuite *)testSuite;

@end

#pragma mark Test Case Implementation

@implementation BTCPreferredOrderIndexGeneratorParameterizedTest

@synthesize generator = _generator;

@synthesize expectedValue = _expectedValue;
@synthesize date = _date;
@synthesize expectedException = _expectedException;

#pragma mark Static ObjC Helper Functions

static NSDate* BTCCreateDate(NSInteger year, NSInteger month, NSInteger day)
{
    NSDateComponents *dateComponents = [[NSDateComponents alloc] init];
    [dateComponents setYear:year];
    [dateComponents setMonth:month];
    [dateComponents setDay:day];

    NSCalendar *calendar = [NSCalendar currentCalendar];
    NSDate *date = [calendar dateFromComponents:dateComponents];

    [dateComponents release];

    return date;
}

#pragma mark Life Cycle Methods

- (id)initWithInvocation:(NSInvocation *)testInvocation
           expectedValue:(unsigned long long)value
                    date:(NSDate *)inputDate
       expectedException:(NSException *)exception {

    self = [super initWithInvocation:testInvocation];
    if (self) {
        [self setExpectedValue:value];
        [self setDate:inputDate];
        [self setExpectedException:exception];
    }

    return self;
}

- (void)setUp
{
    _generator = [[BTCPreferredOrderIndexGenerator alloc] init];
}

- (void)tearDown {
    [self setGenerator:nil];
    [self setDate:nil];
    [self setExpectedException:nil];
}

#pragma mark Class Methods To Generate Test Suite

+ (id)defaultTestSuite
{
    SenTestSuite *testSuite = [[SenTestSuite alloc] initWithName:NSStringFromClass(self)];

    [self addTestWithExpectedValue:2010010100000000000 date:BTCCreateDate(2010, 1, 1) expectedException:nil toTestSuite:testSuite];
    [self addTestWithExpectedValue:2010010900000000000 date:BTCCreateDate(2010, 1, 9) expectedException:nil toTestSuite:testSuite];
    [self addTestWithExpectedValue:2010011000000000000 date:BTCCreateDate(2010, 1, 10) expectedException:nil toTestSuite:testSuite];
    [self addTestWithExpectedValue:2010011100000000000 date:BTCCreateDate(2010, 1, 11) expectedException:nil toTestSuite:testSuite];
    [self addTestWithExpectedValue:2010100900000000000 date:BTCCreateDate(2010, 10, 9) expectedException:nil toTestSuite:testSuite];
    [self addTestWithExpectedValue:2010101000000000000 date:BTCCreateDate(2010, 10, 10) expectedException:nil toTestSuite:testSuite];
    [self addTestWithExpectedValue:2010101100000000000 date:BTCCreateDate(2010, 10, 11) expectedException:nil toTestSuite:testSuite];

    { // nil date should throw an exception
        NSException *expectedException = [[NSException alloc] initWithName:kBTCInvalidDate reason:@"date is nil" userInfo:nil] autoreleased];
        [self addTestWithExpectedValue:-1 date:nil expectedException:expectedException toTestSuite:testSuite];
    }

    return [testSuite autorelease];
}

+ (void)addTestWithExpectedValue:(unsigned long long)expectedValue
                            date:(NSDate *)date
               expectedException:(NSException *)expectedException
                     toTestSuite:(SenTestSuite *)testSuite
{
    NSArray *testInvocations = [self testInvocations];
    for (NSInvocation *testInvocation in testInvocations) {
        SenTestCase *test = [[BTCPreferredOrderIndexGeneratorParameterizedTest alloc]
                initWithInvocation:testInvocation
                     expectedValue:expectedValue
                              date:date
                 expectedException:expectedException];

        [testSuite addTest:test];
        [test release];
    }
}

#pragma mark Test Methods

- (void)testGenerateFromDate
{
    if ([self expectedException] == nil) {
        STAssertEquals(_expectedValue, [_generator generateFromDate:[self date]], @"Generated Preferred Order Index");
    }
    else {
        @try {
            [_generator generateFromDate:_date];
            STFail(@"Generator should raise exception.");
        }
        @catch (NSException *exception) {

            // You should test the exception however it makes sense in your application...
            STAssertEqualObjects([_expectedException class], [exception class], @"Exception Class.");
            STAssertEqualObjects([_expectedException name], [exception name], @"Exception Name.");
            STAssertEqualObjects([_expectedException reason], [exception reason], @"Exception reason.");
        }  
    }
}

@end

Adding Another Test Method

Let's assume that we want to write another test that uses the same input and expected values. This is typically useful if there is another method to test that utilizes the same properties/ ivars. In this article, we will just create an empty test method. The new test method is automatically picked up by our parameterized test case to execute against each set of input data. This is what I mean by a parametized test case is the cross-product between input parameters and test methods.



// . . . original test method omitted from this code snippet

- (void)testSomethingUseful
{
    // do something useful.
}

Test Suite 'BTCPreferredOrderIndexGeneratorTest' started at ... .
Test Case '-[BTCPreferredOrderIndexGeneratorTest testGenerateFromDate]' started.
Test Case '-[BTCPreferredOrderIndexGeneratorTest testGenerateFromDate]' passed (0.000 seconds).
Test Case '-[BTCPreferredOrderIndexGeneratorTest testSomethingUseful]' started.
Test Case '-[BTCPreferredOrderIndexGeneratorTest testSomethingUseful]' passed (0.000 seconds).
Test Case '-[BTCPreferredOrderIndexGeneratorTest testGenerateFromDate]' started.
Test Case '-[BTCPreferredOrderIndexGeneratorTest testGenerateFromDate]' passed (0.000 seconds).
Test Case '-[BTCPreferredOrderIndexGeneratorTest testSomethingUseful]' started.
Test Case '-[BTCPreferredOrderIndexGeneratorTest testSomethingUseful]' passed (0.000 seconds).
Test Case '-[BTCPreferredOrderIndexGeneratorTest testGenerateFromDate]' started.
Test Case '-[BTCPreferredOrderIndexGeneratorTest testGenerateFromDate]' passed (0.000 seconds).
Test Case '-[BTCPreferredOrderIndexGeneratorTest testSomethingUseful]' started.
Test Case '-[BTCPreferredOrderIndexGeneratorTest testSomethingUseful]' passed (0.000 seconds).
Test Case '-[BTCPreferredOrderIndexGeneratorTest testGenerateFromDate]' started.
Test Case '-[BTCPreferredOrderIndexGeneratorTest testGenerateFromDate]' passed (0.000 seconds).
Test Case '-[BTCPreferredOrderIndexGeneratorTest testSomethingUseful]' started.
Test Case '-[BTCPreferredOrderIndexGeneratorTest testSomethingUseful]' passed (0.000 seconds).
Test Case '-[BTCPreferredOrderIndexGeneratorTest testGenerateFromDate]' started.
Test Case '-[BTCPreferredOrderIndexGeneratorTest testGenerateFromDate]' passed (0.000 seconds).
Test Case '-[BTCPreferredOrderIndexGeneratorTest testSomethingUseful]' started.
Test Case '-[BTCPreferredOrderIndexGeneratorTest testSomethingUseful]' passed (0.000 seconds).
Test Case '-[BTCPreferredOrderIndexGeneratorTest testGenerateFromDate]' started.
Test Case '-[BTCPreferredOrderIndexGeneratorTest testGenerateFromDate]' passed (0.000 seconds).
Test Case '-[BTCPreferredOrderIndexGeneratorTest testSomethingUseful]' started.
Test Case '-[BTCPreferredOrderIndexGeneratorTest testSomethingUseful]' passed (0.000 seconds).
Test Case '-[BTCPreferredOrderIndexGeneratorTest testGenerateFromDate]' started.
Test Case '-[BTCPreferredOrderIndexGeneratorTest testGenerateFromDate]' passed (0.000 seconds).
Test Case '-[BTCPreferredOrderIndexGeneratorTest testSomethingUseful]' started.
Test Case '-[BTCPreferredOrderIndexGeneratorTest testSomethingUseful]' passed (0.000 seconds).
Test Case '-[BTCPreferredOrderIndexGeneratorTest testGenerateFromDate]' started.
Test Case '-[BTCPreferredOrderIndexGeneratorTest testGenerateFromDate]' passed (0.000 seconds).
Test Case '-[BTCPreferredOrderIndexGeneratorTest testSomethingUseful]' started.
Test Case '-[BTCPreferredOrderIndexGeneratorTest testSomethingUseful]' passed (0.000 seconds).
Test Suite 'BTCPreferredOrderIndexGeneratorTest' finished at ... .
Executed 16 tests, with 0 failures (0 unexpected) in 0.001 (0.002) seconds

The test case automatically picked up the new test method and added a new test invocation for each set of input values.

Summary

I love these types of tests. I use them all of the time. You should consider writing a "parameterized" test case the next time time you find yourself copying and pasting test methods and changing the values.

Usages/ Examples

  • value transformations
  • calendrical calculators
  • Core Data entity attribute validation
  • any public API method that takes in one or more parameters and returns a calculated result.

Just remember these basic steps when creating an OCUnit "Parameterized" test:

  1. Create a new subclass of SenTestCase (same as any other OCUnit test case)
  2. Override the class method defaultTestSuite to dynamically generate a new SenTestSuite
  3. Add properties/ ivars to the test case that represent the input value(s) and expected output value(s)
  4. Create one or more standard test methods that exercise the input/ expected values.