Here in this tutorial I will tell you what are Dynamic Tests and @TestFactory
in Junit 5 and how to create @TestFactory
in Junit 5.
Test cases, annotated with @Test
, are static in the sense that they are fully specified at compile time, and their behavior cannot be changed by anything happening at runtime.
In addition to these standard static tests a new kind of test programming model has been introduced in JUnit Jupiter. This new kind of test is a dynamic test which is generated at runtime by a factory method that is annotated with @TestFactory
.
Unlike @Test
methods, a @TestFactory
method is not itself a test case but rather a factory for test cases. Therefore, a @TestFactory
method must return a single DynamicNode or a Stream, Collection, Iterable, Iterator, or array of DynamicNode instances.
Instantiable subclasses of DynamicNode are DynamicContainer and DynamicTest. DynamicContainer instances are composed of a display name and a list of dynamic child nodes, enabling the creation of arbitrarily nested hierarchies of dynamic nodes. DynamicTest instances will be executed lazily, enabling dynamic and even non-deterministic generation of test cases.
Any Stream returned by a @TestFactory
will be properly closed by calling stream.close()
, making it safe to use a resource such as Files.lines()
.
@TestFactory
methods must not be private or static and may optionally declare parameters to be resolved by ParameterResolvers
.
A DynamicTest
is a test case generated at runtime. It is composed of a display name and an Executable
. Executable
is a @FunctionalInterface
which means that the implementations of dynamic tests can be provided as lambda expressions or method references.
The execution lifecycle of a dynamic test is quite different than it is for a standard @Test
case. Specifically, there are no lifecycle callbacks for individual dynamic tests. This means that @BeforeEach
and @AfterEach
methods and their corresponding extension callbacks are executed for the @TestFactory
method but not for each dynamic test. In other words, if you access fields from the test instance within a lambda expression for a dynamic test, those fields will not be reset by callback methods or extensions between the execution of individual dynamic tests generated by the same @TestFactory
method.
Now I am going to show you examples on Dynamic Tests in Junit 5.
Prerequisites
Java at least 8, Junit 5.0.8-M1, Maven 3.6.3
Project Setup
Create a maven based project in your favorite IDE or tool. The name of the project is java-junit-dynamic-tests.
Dynamic Tests
The following methods are very simple examples that demonstrate the generation of a Collection, Iterable, Iterator of DynamicTest instances. However, these examples do not really exhibit dynamic behavior but merely demonstrate the supported return types in principle.
@TestFactory
Collection<DynamicTest> dynamicTestsMinMaxPalindromeCollection() {
return Arrays.asList(
dynamicTest("Find Max Test",
() -> assertEquals(4, MinMaxElementFinder.findMax(new int[] { 1, 3, 4, 2 }))),
dynamicTest("Find Min Test",
() -> assertEquals(1, MinMaxElementFinder.findMin(new int[] { 1, 3, 4, 2 }))),
dynamicTest("Palindrome 1", () -> assertTrue(checker.isPalindrome("madam"))),
dynamicTest("Palindrome 2", () -> assertFalse(checker.isPalindrome("palindrome"))));
}
@TestFactory
Iterable<DynamicTest> dynamicTestsMinMaxPalindromeIterable() {
return Arrays.asList(
dynamicTest("Find Max Test",
() -> assertEquals(4, MinMaxElementFinder.findMax(new int[] { 1, 3, 4, 2 }))),
dynamicTest("Find Min Test",
() -> assertEquals(1, MinMaxElementFinder.findMin(new int[] { 1, 3, 4, 2 }))),
dynamicTest("Palindrome 1", () -> assertTrue(checker.isPalindrome("madam"))),
dynamicTest("Palindrome 2", () -> assertFalse(checker.isPalindrome("palindrome"))));
}
@TestFactory
Iterator<DynamicTest> dynamicTestsMinMaxPalindromeIterator() {
return Arrays.asList(
dynamicTest("Find Max Test",
() -> assertEquals(4, MinMaxElementFinder.findMax(new int[] { 1, 3, 4, 2 }))),
dynamicTest("Find Min Test",
() -> assertEquals(1, MinMaxElementFinder.findMin(new int[] { 1, 3, 4, 2 }))),
dynamicTest("Palindrome 1", () -> assertTrue(checker.isPalindrome("madam"))),
dynamicTest("Palindrome 2", () -> assertFalse(checker.isPalindrome("palindrome")))).iterator();
}
@TestFactory
DynamicTest[] dynamicTestsMinMaxPalindromeArray() {
return new DynamicTest[] {
dynamicTest("Find Max Test",
() -> assertEquals(4, MinMaxElementFinder.findMax(new int[] { 1, 3, 4, 2 }))),
dynamicTest("Find Min Test",
() -> assertEquals(1, MinMaxElementFinder.findMin(new int[] { 1, 3, 4, 2 }))),
dynamicTest("Palindrome 1", () -> assertTrue(checker.isPalindrome("madam"))),
dynamicTest("Palindrome 2", () -> assertFalse(checker.isPalindrome("palindrome"))) };
}
The following methods demonstrate how easy it is to generate dynamic tests for a given set of strings or a range of input numbers.
@TestFactory
Stream<DynamicTest> dynamicTestsStream() {
return Stream.of("madam", "mom", "dad")
.map(str -> dynamicTest(str, () -> assertTrue(checker.isPalindrome(str))));
}
@TestFactory
Stream<DynamicTest> dynamicTestsIntStream() {
return IntStream.iterate(0, n -> n + 2).limit(5)
.mapToObj(n -> dynamicTest("Even Test" + n, () -> assertTrue(n % 2 == 0)));
}
@TestFactory
Stream<DynamicTest> dynamicTestsStreamFactoryMethod() {
Stream<String> inputStream = Stream.of("madam", "mom", "dad");
Function<String, String> displayNameGenerator = str -> str + " is a palindrome";
ThrowingConsumer<String> testExecutor = str -> assertTrue(checker.isPalindrome(str));
return DynamicTest.stream(inputStream, displayNameGenerator, testExecutor);
}
The following method generates a single DynamicTest instead of a stream:
@TestFactory
DynamicNode dynamicNodeSingle() {
return dynamicTest("'push' is a not palindrome", () -> assertFalse(checker.isPalindrome("push")));
}
The following method generates a nested hierarchy of dynamic tests utilizing DynamicContainer:
@TestFactory
DynamicNode dynamicNodeSingleContainer() {
return dynamicContainer("Palindromes", Stream.of("madam", "mom", "dad")
.map(str -> dynamicTest(str, () -> assertTrue(checker.isPalindrome(str)))));
}
URI Test Sources for Dynamic Tests
The TestSource for a dynamic test or dynamic container can be constructed from a java.net.URI which can be supplied via the DynamicTest.dynamicTest(String, URI, Executable) or DynamicContainer.dynamicContainer(String, URI, Stream) factory method, respectively. The URI will be converted to one of the following TestSource implementations.
ClasspathResourceSource: If the URI contains the classpath scheme — for example, classpath:/test/foo.xml?line=20,column=2.
DirectorySource: If the URI represents a directory present in the file system.
FileSource: If the URI represents a file present in the file system.
MethodSource: If the URI contains the method scheme and the fully qualified method name (FQMN) — for example, method:org.junit.Foo#bar(java.lang.String, java.lang.String[]). Please refer to the Javadoc for DiscoverySelectors.selectMethod(String) for the supported formats for a FQMN.
UriSource: If none of the above TestSource implementations are applicable.
The example on URI for Test Sources for Dynamic Tests can be written as follows:
@TestFactory
Stream<DynamicTest> checkAllTextFiles() throws Exception {
return Files.walk(Paths.get("src/test/resources/test"), 1).filter(path -> path.toString().endsWith(".txt"))
.map(path -> dynamicTest("File: " + path.getFileName(), path.toUri(), () -> checkLineContent(path)));
}
private void checkLineContent(Path path) throws Exception {
List<String> lines = Files.readAllLines(path);
String expected = lines.get(0);
String actual = new StringBuilder(lines.get(0)).reverse().toString();
assertEquals(expected, actual, "String and its reverse should be equal");
}
The whole Junit class can be written as follows:
package com.roytuts.java.junit.dynamic.tests;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.junit.jupiter.api.DynamicContainer.dynamicContainer;
import static org.junit.jupiter.api.DynamicTest.dynamicTest;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Arrays;
import java.util.Collection;
import java.util.Iterator;
import java.util.List;
import java.util.function.Function;
import java.util.stream.IntStream;
import java.util.stream.Stream;
import org.junit.jupiter.api.DynamicNode;
import org.junit.jupiter.api.DynamicTest;
import org.junit.jupiter.api.TestFactory;
import org.junit.jupiter.api.function.ThrowingConsumer;
public class JunitDynamicTests {
private final PalindromeChecker checker = new PalindromeChecker();
@TestFactory
Collection<DynamicTest> dynamicTestsMinMax() {
return Arrays.asList(
dynamicTest("Find Max Test",
() -> assertEquals(4, MinMaxElementFinder.findMax(new int[] { 1, 3, 4, 2 }))),
dynamicTest("Find Min Test",
() -> assertEquals(1, MinMaxElementFinder.findMin(new int[] { 1, 3, 4, 2 }))));
}
@TestFactory
Collection<DynamicTest> dynamicTestsPalindrome() {
return Arrays.asList(dynamicTest("Palindrome 1", () -> assertTrue(checker.isPalindrome("madam"))),
dynamicTest("Palindrome 2", () -> assertFalse(checker.isPalindrome("palindrome"))));
}
@TestFactory
Collection<DynamicTest> dynamicTestsMinMaxPalindromeCollection() {
return Arrays.asList(
dynamicTest("Find Max Test",
() -> assertEquals(4, MinMaxElementFinder.findMax(new int[] { 1, 3, 4, 2 }))),
dynamicTest("Find Min Test",
() -> assertEquals(1, MinMaxElementFinder.findMin(new int[] { 1, 3, 4, 2 }))),
dynamicTest("Palindrome 1", () -> assertTrue(checker.isPalindrome("madam"))),
dynamicTest("Palindrome 2", () -> assertFalse(checker.isPalindrome("palindrome"))));
}
@TestFactory
Iterable<DynamicTest> dynamicTestsMinMaxPalindromeIterable() {
return Arrays.asList(
dynamicTest("Find Max Test",
() -> assertEquals(4, MinMaxElementFinder.findMax(new int[] { 1, 3, 4, 2 }))),
dynamicTest("Find Min Test",
() -> assertEquals(1, MinMaxElementFinder.findMin(new int[] { 1, 3, 4, 2 }))),
dynamicTest("Palindrome 1", () -> assertTrue(checker.isPalindrome("madam"))),
dynamicTest("Palindrome 2", () -> assertFalse(checker.isPalindrome("palindrome"))));
}
@TestFactory
Iterator<DynamicTest> dynamicTestsMinMaxPalindromeIterator() {
return Arrays.asList(
dynamicTest("Find Max Test",
() -> assertEquals(4, MinMaxElementFinder.findMax(new int[] { 1, 3, 4, 2 }))),
dynamicTest("Find Min Test",
() -> assertEquals(1, MinMaxElementFinder.findMin(new int[] { 1, 3, 4, 2 }))),
dynamicTest("Palindrome 1", () -> assertTrue(checker.isPalindrome("madam"))),
dynamicTest("Palindrome 2", () -> assertFalse(checker.isPalindrome("palindrome")))).iterator();
}
@TestFactory
DynamicTest[] dynamicTestsMinMaxPalindromeArray() {
return new DynamicTest[] {
dynamicTest("Find Max Test",
() -> assertEquals(4, MinMaxElementFinder.findMax(new int[] { 1, 3, 4, 2 }))),
dynamicTest("Find Min Test",
() -> assertEquals(1, MinMaxElementFinder.findMin(new int[] { 1, 3, 4, 2 }))),
dynamicTest("Palindrome 1", () -> assertTrue(checker.isPalindrome("madam"))),
dynamicTest("Palindrome 2", () -> assertFalse(checker.isPalindrome("palindrome"))) };
}
@TestFactory
Stream<DynamicTest> dynamicTestsStream() {
return Stream.of("madam", "mom", "dad")
.map(str -> dynamicTest(str, () -> assertTrue(checker.isPalindrome(str))));
}
@TestFactory
Stream<DynamicTest> dynamicTestsIntStream() {
return IntStream.iterate(0, n -> n + 2).limit(5)
.mapToObj(n -> dynamicTest("Even Test" + n, () -> assertTrue(n % 2 == 0)));
}
@TestFactory
Stream<DynamicTest> dynamicTestsStreamFactoryMethod() {
Stream<String> inputStream = Stream.of("madam", "mom", "dad");
Function<String, String> displayNameGenerator = str -> str + " is a palindrome";
ThrowingConsumer<String> testExecutor = str -> assertTrue(checker.isPalindrome(str));
return DynamicTest.stream(inputStream, displayNameGenerator, testExecutor);
}
@TestFactory
Stream<DynamicNode> dynamicTestsContainers() {
return Stream.of("Roytuts", "Soumitra", "Roy").map(input -> dynamicContainer("Container " + input,
Stream.of(dynamicTest("Not Null", () -> assertNotNull(input)),
dynamicContainer("Conditional Checks",
Stream.of(dynamicTest("Empty", () -> assertNull(null)),
dynamicTest("False", () -> assertFalse(false)),
dynamicTest("Length Greater Than 0", () -> assertTrue(input.length() > 0)),
dynamicTest("Not Empty", () -> assertFalse(input.isEmpty())))))));
}
@TestFactory
DynamicNode dynamicNodeSingle() {
return dynamicTest("'push' is a not palindrome", () -> assertFalse(checker.isPalindrome("push")));
}
@TestFactory
DynamicNode dynamicNodeSingleContainer() {
return dynamicContainer("Palindromes", Stream.of("madam", "mom", "dad")
.map(str -> dynamicTest(str, () -> assertTrue(checker.isPalindrome(str)))));
}
@TestFactory
Stream<DynamicTest> checkAllTextFiles() throws Exception {
return Files.walk(Paths.get("src/test/resources/test"), 1).filter(path -> path.toString().endsWith(".txt"))
.map(path -> dynamicTest("File: " + path.getFileName(), path.toUri(), () -> checkLineContent(path)));
}
private void checkLineContent(Path path) throws Exception {
List<String> lines = Files.readAllLines(path);
String expected = lines.get(0);
String actual = new StringBuilder(lines.get(0)).reverse().toString();
assertEquals(expected, actual, "String and its reverse should be equal");
}
}
The whole source code can be downloaded from the Source Code section later in the below.
Testing Dynamic Tests
Running the Junit class will give you the following output:
