Swift Parameterized XCTest Test Case

NSInvocation is not yet allowed in Swift. This means that any existing ObjC code using NSInvocation must be re-designed if you want to use Swift. Fortunately we don't have to completely re-design our existing parameterized test cases. In fact, as we will see, it is really super simple to convert ObjC parameterized tests to Swift parameterized tests.

Download the sample project on GitHub.

Overview

A parameterized, or data-driven, test case allows developers to create one test method that executes N number of times. This technique works because each test executes using a unique instance of the test case.

The example described here is similar to the technique used in the Java Extreme Programming Cookbook, which I co-authored back in 2002.

I also have an older ObjC+SenTestKit post that provides additional detail.

Let's Build It

A parameterized test case is a standard test case (i.e. a subclass of XCTestCase), usually containing a single test method that executes multiple times with a different set of input values and expected values.

There are five main steps:

1. Create a new subclass of XCTestCase (same as any other XCUnit test case)
2. Add properties to the test case that represent the input value(s) and expected output value(s)
3. Override the class method defaultTestSuite to manually create a new XCTestSuite
4. Call testInvocations() for each set of input data
5. Create one or more standard test methods that exercise the input/ expected values.

But NSInvocation Is Not Available In Swift

Parameterized tests rely on a XCTestCase class level method named testInvocations() -> [NSInvocation]. This method knows how to scan the test case looking for methods that begin with test, take zero arguments and return void.

Swift, however, does not support NSInvocation. So what do we do?

We can take advantage of a "loop hole". As long as we don't include the

NSInvocation symbol in our code, then the Swift compiler will not complain. The code executes just fine at runtime.

Sample Test Case

The sample test case is extremely contrived. The test case contains two private properties:

1. input array of strings
2. expected string based on the input array of strings

class SomeSampleTest: XCTestCase {

    // MARK: Private Properties

    private let array: [String]
    private let expectedString: String

    // MARK: Object Life Cycle

    // This is what we want to do... but can't
    init(array: [String], expectedString: String, invocation: NSInvocation) {
        super.invocation(invocation)

        self.array = array
        self.expectedString = expectedString
    }
}

The above code won't compile because of NSInvocation. For now let's keep going.

Here's the single "parameterized" test method.

     // MARK: Test Methods

    func testGeneratedString() {

        // Normally you would be testing production code (e.g. class, struct, etc).
        let actualString = self.array.reduce("", combine: +)
        XCTAssertEqual(self.expectedString, actualString)
    }

Now let's build the XCTestSuite

     // MARK: Dynamic Test Creation

    override class func defaultTestSuite() -> XCTestSuite {

        let testSuite = XCTestSuite(name: NSStringFromClass(self))

        addTestsWithArray([], expectedString: "", toTestSuite: testSuite)
        addTestsWithArray(["Brian"], expectedString: "Brian", toTestSuite: testSuite)
        addTestsWithArray(["Brian", "Coyner"], expectedString: "BrianCoyner", toTestSuite: testSuite)
        addTestsWithArray(["Brian", "Coyner", "Was"], expectedString: "BrianCoynerWas", toTestSuite: testSuite)
        addTestsWithArray(["Brian", "Coyner", "Was", "Here"], expectedString: "BrianCoynerWasHere", toTestSuite: testSuite)

        // add more as needed

        return testSuite
    }

    // private helper method to generate tests based on the actual test methods
    // discovered in the test case
    private class func addTestsWithArray(array: [String], expectedString: String, toTestSuite testSuite: XCTestSuite) {

        // Returns an array of NSInvocation objects
        let invocations = self.testInvocations()
        for invocation in invocations {

            // remember this won't compile (yet)
            let testCase = SomeSampleTest(array: array, expectedString: expectedString, invocation: invocation)
            testSuite.addTest(testCase)
        }
    }

Ok. That is the basic path we want to take.

Making It Work

Make the properties Optionals

     // MARK: Private Properties

    private var array: [String]?
    private var expectedString: String?

Use Two Step Initialization

     private class func addTestsWithArray(array: [String], expectedString: String, toTestSuite testSuite: XCTestSuite) {

        // Returns an array of NSInvocation
        let invocations = self.testInvocations()
        for invocation in invocations {

            // We can't directly use the NSInvocation type in our source, but it appears
            // that we can pass it on through to the super class.
            let testCase = SomeSampleTest(invocation: invocation)

            // Set the parameterized on the newly created test case instance
            testCase.array = array
            testCase.expectedString = expectedString

            testSuite.addTest(testCase)
        }
    }

Build and run! It works!

The compiler won't allow us to include the NSInvocation symbol. But we can just pass the invocation instance directly to the test case initializer without ever explicitly typing it as an NSInvocation. How wonderful!

And, of course, don't forget to set the properties. That's the bad part of two step initialization.

Wrapping Up

The NSInvocation workaround turns out to be really simple:

• make private properties used by test methods Optionals
• pass the NSInvocation instance to the super class initializer (but without including the NSInvocation symbol in the code)
• set the properties on the test case instance
• build and run

I tested this on Xcode 7.1 and Xcode 7.2 beta.

Perhaps a future Swift version will allow direct use of NSInvocation, or perhaps Apple will include a better way to create parameterized (data-driven) tests.

This is a huge win for me because I have thousands and thousands of parameterized tests.