Blog

Conditionally ignoring JUnit tests

02 Oct, 2012
Xebia Background Header Wave

A useful technique that I reinvent every once in a while is conditionally ignoring JUnit tests. Unit tests are supposed to be isolated, but occasionally you hit something that makes assumptions about the environment, such as code that executes a platform-specific shell command or (more commonly) an integration test that assumes the presence of a database. To keep such a test from breaking unsuspecting builds, you can @Ignore it, but that means you have to edit the code to run the test in a supported environment.

Proper Maven projects put their integration tests in a separate source folder called src/it/java and put an extra execution of the maven-surefire-plugin into their pom.xml, tied to the integration-test phase of the Maven build lifecycle. This is Maven’s recommended way of setting these up. It ties in beautifully with the pre-integration-test and post-integration-test phases that can be used to set up and tear down the environmental dependencies of the integration test suite, such as initializing a database to a known state. There is nothing wrong with this approach, but it’s a bit heavy handed for the simplest of cases.

In these simple situations it’s easier to just keep the integration tests in the src/test/java directory and run them along with all your other tests. However, you still need a way to trigger them only when the right environment is present. This is easily dealt with by writing your own JUnit TestRunner and some custom annotations, as shown below.

The Entry Point

The default test runner in JUnit 4 is BlockJUnit4ClassRunner (javadoc, source code). From this class we need to override just one method:

[sourcecode language=”java”]
protected void runChild(FrameworkMethod method, RunNotifier notifier)
[/sourcecode]

This method is run once for each test and checks whether the @Ignore annotation is present. Let’s add an annotation of our own, where we run a class only if a certain Java system property is set.

The Method Annotation

We’ll define a simple annotation that targets the method level and defines the system property that must be set for that test method to run:

[sourcecode language=”java”]
@Target( ElementType.METHOD )
@Retention(RetentionPolicy.RUNTIME)
public @interface SystemPropertyCondition {
/** The name of a system property that must be set for the test to run. */
String value();
}
[/sourcecode]

The Test Runner

Writing the test runner is similarly easy:

[sourcecode language=”java”]
public class ConditionalTestRunner extends BlockJUnit4ClassRunner {
public ConditionalTestRunner(Class klass) {
super(klass);
}
@Override
public void runChild(FrameworkMethod method, RunNotifier notifier) {
SystemPropertyCondition condition =
method.getAnnotation(SystemPropertyCondition.class)
if (condition != null && System.getProperty(condition.value()) != null) {
super.runChild(method, notifier);
} else {
notifier.fireTestIgnored(describeChild(method));
}
}
}
[/sourcecode]

The Usage Example

To use this test runner, annotate your test class with the @RunWith(Class) annotation. Any of the test methods can then be annotated to be conditional.

[sourcecode language=”java”]
@RunWith(ConditionalTestRunner.class)
public class SomeExampleTest {
@Test
@SystemPropertyCondition("com.mycompany.includeConditionalTests")
public void testMethodThatRunsConditionally() {
// Normal test code goes here
}
}
[/sourcecode]

This approach can easily be extended for greater flexibility. For example, if you want to be able to annotate entire classes as well as individual methods, make the annotation target both ElementType.TYPE and ElementType.METHOD and, in your test runner, evaluate not only method.getAnnotations(), but also getTestClass().getAnnotations(). It’s not difficult to add extra annotation types that check for things like Host OS, environment variables or an open TCP port on localhost.

The one thing to keep in mind is that each and every one of these conditionals that you add to your code violates the ideal situation where unit tests can run in any environment and any order. Apply judiciously.

Questions?

Get in touch with us to learn more about the subject and related solutions

Explore related posts