Refactoring Catalogue

Lazily Evaluate Values

Assertion Refactorings

Motivation

Lazily evaluated attributes and values are only computed when they are needed.

In tests, some components are good candidates for being lazily evaluated, i.e., SUT dependencies and assertions.

For instance, assertions' message of failure that are expensive to compute may benefit from lazy evaluation.

Qualities

Efficiency

Code Demonstration

BEFORE

assertEquals(expected, actual, message: slowComputation())

AFTER

message = () -> slowComputation()
assertEquals(expected, actual, message)

Instances

Sources

StackOverflow

Consolidate Multiple Assertions into a Fluent Assertion

Assertion Refactorings

Motivation

Test cases with many assertions are known as the assertion roulette test smell.

One way to fix it is replacing all assertions that have the same actual value with one single Fluent Assertion.

Fluent Assertions enables chaining together different condition checks as unary method invocations.

In comparison, traditional assertions take up to three parameters, therefore they are subject to confusion, e.g., failing a binary assertion with a custom message is expressed differently in JUnit 4 assertEquals(message, expected, actual), JUnit 5 assertEquals(expected, actual, message), and TestNG assertEquals(actual, expected, message)

Conversely, in AssertJ, the same assertion would look like assertThat(actual).withFailMessage(message).isEqualTo(expected).

Qualities

Uniformity

Code Demonstration

BEFORE

assertEquals(9, actual.size())
assertTrue(actual.contains(element))

AFTER

assertThat(actual)
  .hasSize(9)
  .contains(expected)

Instances

Sources

StackOverflowMining Datasets

Group Multiple Assertions

Assertion Refactorings

Motivation

Test Methods with multiple assertions may have its execution interrupted prematurely, thus preventing the evaluation of the remaining assertions (Soares et al., 2023).

Grouping multiple assertions (a₁, a₂,...,aₙ) with JUnit 5's assertAll(()->a₁, ()->a₂,..., ()->aₙ) assures that they execute and their result are outputted to the test report before terminating the test suite execution.

Qualities

Effectiveness

Code Demonstration

BEFORE

assert(a₁)
...
assert(aₙ)

AFTER

AssertAll(
  () -> assert(a₁)
  ...
  () -> assert(aₙ)
)

Instances

Commit: Thunder

Sources

Mining DatasetsLiterature (Soares et al., 2023)

Replace Assertion

Assertion Refactorings

Variants

1. Replace the Not Operator2. Split Conditional Parameters3. Replace Reserved Words

Motivation

1. Having a conditional logic in the assertions using the not (!) operator affect their readability.

2. Forced conditional expressions into assertTrue/assertFalse methods to verify objects equality should be replaced.

3. Developers should use special assertions (e.g. assertNull) instead of passing reserved words.

Qualities

Uniformity
Effectiveness
Simplicity

Code Demonstration

BEFORE

assert(!condition)
assert(actual == expected)
assert(actual == true)
assert(actual == null)

AFTER

assertFalse(condition)
assertEquals(expected, actual)
assertTrue(actual)
assertNull(actual)

Instances

Sources

StackOverflowLiterature (Martins et al., 2023)

Enforce Use of Type Inference

Assertion Refactorings

Motivation

As shown by Kashiwa et al. (2021) Change Return Type and Add Parameter refactorings in production APIs are mainly responsible for breaking tests, and require 4-12 lines of changes to fix the tests they break.

A good practice to avoid or limit the impact of such breaking changes in tests is to utilize the var keyword (Java 10 feature) in the place of type references within the tests.

When the var keyword is used, JVM applies type inference to detect automatically the datatype of a variable based on the surrounding context.

This approach makes the tests more resilient to SUT type-related changes.

Qualities

Brittleness

Code Demonstration

BEFORE

Type variable = new ConcreteType()
AnotherType result = methodCall()
List<SpecificType> list = getItems()

AFTER

var variable = new ConcreteType()
var result = methodCall()
var list = getItems()

Instances

Commit: Camel

Sources

Monitoring

Abstract away SUT's Construction

Fixture Refactorings

Variants

1. AutoFixture2. Object Mother3. Test Data Builder with Fluent API

Motivation

1. C# only, AutoFixture works as a generic Test Data Builder and an Auto-Mocking container (SO 2056602) built upon sensible defaults for different attribute types, e.g., DateTime.Now (SO 2622334).

2. Sometimes the same hard-to-build objects can be reused throughout the test suite; ObjectMother (Schuh and Punke, 2001; Fields, 2014) encapsulates the complex build logic into its static methods for reuse and code clone elimination.

3. Unlike ObjectMother, "Test Data Builder copes well with variation in test data" (Fields, 2014). By providing a customizable test object creation with sensible defaults, Test Data Builder favours transparency over conciseness.

Qualities

Changeability
Uniformity
Reliability
Brittleness
Reusability
Efficiency

Code Demonstration

BEFORE

@Test  
void t() {  
    SUT sut = new SUT(new ComplexDependency(
        new NestedDependency(...),  
    ...))  
}

AFTER

// Variant (1) - C# only
[Test]  
void t() {  
    Fixture fixture = new Fixture()
    SUT sut = fixture.Create<SUT>()
}  

// Variant (2)
@Test  
void t() {  
    SUT sut = ObjectMother.CreateDefaultSUT()  
} 

// Variant (3)
@Test  
void t() {  
    SUT sut = new SutBuilder()  
        .WithParam1(expression)  
        .WithParam2(expression)  
        .Build()  
}

Instances

Sources

MonitoringStackOverflowLiterature (Schuh and Punke, 2001; Fields, 2014)

Extract Preconditions

Fixture Refactorings

Variants

1. Extract Preconditions to Class-level Fixture2. Extract Custom Utility Assumption Method3. Replace Branch with Guard Assertion4. Replace Branch with Conditional Execution5. Guard tests with try-catch-assume

Motivation

Preconditions can either be expressed through special assertions (e.g., JUnit 5 assumptions) or regular ones.

It all comes down to whether an unmet precondition should be interpreted as a test failure or not as JUnit 5 assumptions behave similarly to assertions except they abort tests without failing them.

Preconditions are a set of rules to determine whether it makes sense to continue the execution of specific test methods.

1. Placing preconditions in a class-level fixture is particularly useful for efficiency as it aborts the execution of all tests in a test class as soon as the preconditions fail once.

2. Custom Utility Assumption Methods assist developers keeping a consistent mechanism to skip specific test methods in a test class.

3. Tests can contain code that may or may not be executed due to branching logic. The branching logic may cause the test to finish without executing the intended assertion(s). Adding guard assertions (Meszaros, 2007) to the beginning of tests guarantee the test code will only execute when established preconditions are met.

4. Conditional Execution (Soares et al., 2023), introduced in JUnit 5, enables test skipping according to programmatic conditions. Unlike guard assertions, Conditional Execution are expressed as method annotations, which further separate test logic from its preconditions.

5. When conditions are not clear or easily computed, expecting exception could be a valid alternative. Try-catch statement with failing assumptions in the catch-block may be leveraged to make exceptions disable a test rather than failing or passing it. Such a workaround compensates for the lack of an Assumptions.doesNotThrow API in JUnit.

Qualities

Efficiency
Reliability
Uniformity

Code Demonstration

BEFORE

@Test  
void t() {  
    if (C) {  
        stmt
    }
}

AFTER

// Variant (1)
@BeforeAll  
static void setUpClass() {  
    assumeTrue(C) 
}  

@Test  
void t() {  
    stmt  
}  

// Variant (2)
@Test  
void t() {  
    assumePrecondition() 
    stmt  
}  

// Variant (3)
@Test  
void t() {  
    assertTrue(C)
    stmt  
}  

// Variant (4)
@Test  
@EnabledIf("C") 
void t() {  
    stmt  
}  

// Variant (5): Provided stmt throws E when C is false
@Test  
void t() {  
	try {
		stmt
	} catch (E) {
		assumeTrue(false)
	}
}

Instances

Sources

MonitoringStackOverflowLiterature (Meszaros, 2007; Soares et al., 2023)

Reuse code with Fixture / Extract Fixture

Fixture Refactorings

Motivation

Test methods within a single test class may evolve and share many code similarities.

In that case, unless the duplication only affects few methods in the test class, it is advisable to place the duplicated code into a test fixture, e.g., set-up or tear-down methods.

The former comprises test data creation and configuration, whereas the latter should clean up the environment and destroy test objects safely.

If the desired test fixture type already exists in the test class, one can reuse it.

Otherwise, one extracts a new fixture containing the duplicated code.

Qualities

Reliability
Uniformity
Reusability
Changeability
Simplicity

Code Demonstration

BEFORE

@Test  
void t1() {  
    setUp
    stmt
    teardown
}

@Test  
void t2() {  
    setUp
    stmt'
    teardown
}  

@AfterEach
void tearDown() {
    teardown'
}

AFTER

// Extract Fixture
@BeforeEach  
void setUp() {  
    setUp
} 

// Reuse Code with Fixture
@AfterEach  
void tearDown() {
    teardown  
    teardown'
}  

@Test  
void t1() {  
    stmt 
} 

@Test  
void t2() {  
    stmt'
}

Instances

Sources

Mining DatasetsMonitoringStackOverflow

Inline Fixture

Fixture Refactorings

Motivation

As new methods are added and others removed, test fixtures may become obsolete and less cohesive, i.e., General Fixture test smell.

As a consequence, one may inline the test fixture, introducing code duplication into the few test methods that depend on it.

Qualities

Changeability
Independence

Code Demonstration

BEFORE


@BeforeEach  
void setUp() {  
    setUp
} 

@AfterEach  
void tearDown() {  
    teardown
}  

@Test  
void t1() {  
    stmt  
}  

@Test  
void t2() {  
    stmt'  
}

AFTER


@Test  
void t1() {  
    setUp
    stmt
    teardown 
}  

@Test  
void t2() {  
    setUp
    stmt'
    teardown 
}

Instances

Sources

Mining DatasetsStackOverflow

Minimize Fixture

Fixture Refactorings

Motivation

Diverging context of test methods in a single class leads to a General Fixture, i.e., a fixture whose statements inconsistently target different subsets of the test methods.

"Minimizing fixtures make tests better suitable as documentation and less sensitive to changes" (Deursen et al., 2001)

Qualities

Independence

Code Demonstration

BEFORE


@BeforeEach  
void setUp() {  
    setUp
    setUp'  
    setUp''
}

@Test
void t1() {
    stmt
}

@Test
void t2() {
    stmt'
}

AFTER

@BeforeEach
void setUp() {
    setUp
}

@Test
void t1() {
    setUp'
    stmt
}

@Test
void t2() {
    setUp''
    stmt'
}

Instances

Sources

Mining DatasetsStackOverflowLiterature (Meszaros, 2007; Deursen et al., 2001)

Override Fixtures

Fixture Refactorings

Motivation

Despite the risk of introducing redundancy and inter-test dependency, inheritance is extensively used in Java tests.

For reusing tests under different environment settings, testers extract subclasses that override test fixtures.

However, JUnit 5 changed how overriding test fixtures are executed.

While in JUnit 4 test fixtures implemented in subclasses are always executed, JUnit 5 requires the subclass's fixture to be annotated with one of the test fixture annotations (i.e., @BeforeEach, @BeforeAll, @AfterEach, or @AfterAll) even when that particular test fixture overrides a test fixture in the superclass that is annotated as such (Travis, 2018).

Qualities

Changeability
Reusability

Code Demonstration

BEFORE

class C1 {
    @BeforeEach  
    void setUp() {  
        setUp      
    }
} 

class C2 {
    void setUp() {  
        setUp              
        setUp'  
    }  
}

AFTER

class C1 {
    @BeforeEach  
    void setUp() {  
        setUp 
    }  
}

class C2 extends C1 {
    @Override  
    @BeforeEach  
    void setUp() {
        super.setUp()
        setUp'  
    }  
}

Instances

Sources

MonitoringStackOverflow

Inject SUT Dependency

Fixture Refactorings

Variants

1. Constructor-based Dependency2. Replace Fixture with Dependency Injection

Motivation

1. Complex SUT dependencies are good mocking candidates: injecting such dependencies into the SUT constructor often enables to evaluate the interaction with the mocked dependency.

2. Some dependency injection containers, e.g., Spring and Guice, support constructor injection, but test classes usually have a setup method (a test fixture) rather than a constructor. To take advantage of Dependency Injection, your test fixture should be replaced.

Qualities

Reliability
Uniformity
Brittleness

Code Demonstration

BEFORE

class C {
    @BeforeEach
    void setUp() {
    	var dep = new DependencyImpl()
        sut = new SUT(dep)     
    }
}

AFTER

// Variant (1)
class C {
    @BeforeEach
    void setUp() {
        Dependency dep = mock(Dependency.class)  
        SUT sut = new SUT(dep)  
    }
}

// Variant (2)
class C {
    @Autowired
    C(Dependency dep) {
        sut = new SUT(dep)     
    }
}

Instances

Sources

Mining DatasetsStackOverflowLiterature (Meszaros, 2007)

Replace Class Fixture with Method Fixture

Fixture Refactorings

Motivation

The components defined in a test fixture should not be modified during the test execution unless the test fixture runs once per test method.

Qualities

Independence

Code Demonstration

BEFORE

@BeforeAll
void setUp() { }

AFTER

@BeforeEach
void setUp() { }

Instances

Sources

Mining DatasetsStackOverflowLiterature (Kim et al., 2021)

Replace Method Fixture with Class Fixture

Fixture Refactorings

Motivation

Sometimes a test fixture does not have to execute once for each test method, thus, constraining its execution to once for all methods in a test suite can speed-up test execution.

Qualities

Efficiency

Code Demonstration

BEFORE

@BeforeEach
void setUp() { }

AFTER

@BeforeAll
void setUp() { }

Instances

Sources

Mining DatasetsStackOverflowLiterature (Kim et al., 2021)

Split Fixture

Fixture Refactorings

Motivation

When performance is a concern, testers may prefer a setupClass method rather than a setup method, but there is a caveat, not all statements are safe for setupClass.

Then, developers can split the time-consuming part of a fixture, which goes into setupClass, from the part that require execution before each test, which goes into setup.

Qualities

Efficiency

Code Demonstration

BEFORE

@BeforeEach
void setUp() {
    stmt
    stmt'
}

AFTER

@BeforeAll
void setUpClass() {
    stmt
}

@BeforeEach
void setUp() {
    stmt'
}

Instances

Sources

Mining DatasetsStackOverflow

Merge Fixture

Fixture Refactorings

Motivation

Merge separate fixture methods to simplify setup logic when separation is unnecessary.

Qualities

Changeability
Simplicity

Code Demonstration

BEFORE

@BeforeAll
void setUpClass() {
    stmt
}

@BeforeEach
void setUp() {
    stmt'
}

AFTER

@BeforeEach
void setUp() {
    stmt
    stmt'
}

Instances

Sources

Mining Datasets

Parameterize Test with Framework Support

Test Method Refactorings

Variants

1. Extract Common Logic from Multiple Test Methods2. Merge Data Provider3. Multiple Assertion Type Test become Conditional Parameterized Test4. Multiple Data and Multiple Algorithms become Parameterized Test with Inheritance and Fixture Overrides

Motivation

1. The logic of many test methods may share similarities, i.e., test code duplication. When multiple methods only differ in terms of data input, it is a good candidate for parameterization.

2. JUnit 5’s and TestNG’s parameterized test class have at least one data provider, but having many may compromise the tests’ cohesion and maintainability.

3. To accommodate variation in the reuse of mostly similar test methods, parameterized tests may include flag parameters to enable/disable test methods with assumptions or assertions guarded by if-statements.

4. When multiple implementations with similar interfaces and behaviour (e.g., sorting algorithm) must be tested for multiple data, one may parameterize the test and extract test subclasses to build each implementation in a setup fixture.

Qualities

Reusability
Reliability
Uniformity

Code Demonstration

BEFORE

// Variant (1)
@Test  
void t1n() {  
    stmt¹ⁿ(P₁ₙ) 
    ...
    stmt¹ⁿ(P₁ₙ)
} 
...
@Test  
void t2() {  
    stmtᵐ¹(Pₘ₁) 
    ...
    stmtᵐⁿ(Pₘₙ)
}  

// Variant (2)
@CsvSource({
    P₁₁, ..., P₁ₙ
})  
void t1() { } 

@CsvSource({
    Pₘ₁, ..., Pₘₙ
})  
void t2() { }

AFTER

// Variant (1) and (2)
@ParameterizedTest  
@CsvSource({
    P₁₁, ..., P₁ₙ
    ...
    Pₘ₁, ..., Pₘₙ
})    
void t(P₁, ..., Pₙ) {
    stmt¹(P₁)
    ...
    stmtⁿ(Pₙ)
}

BEFORE

void t() {
    List actual = sut.method()
    assertEquals(expcted, actual)
}
void t2() {
   int[] actual = sut.method()
   assertArrayEquals(expcted, actual)
}

AFTER

@ParameterizedTest  
@MethodSource 
void t(SUT sut) {
    actual = sut.method()
    if (P₁ instanceof P) {
        assertEquals(expcted, actual)    
    }
    else {
        assertArrayEquals(expcted, actual)
    }        
    ...
    assumeTrue(Pₙ)
    stmtⁿ(Pₙ)
}

BEFORE

class AlgorithmATest {
    Algorithm algo = new AlgorithmA()
    
    @Test
    void t() { 
        algo.process(d')
    }

    @Test
    void t2() { 
        algo.process(d'')
    }
}

class AlgorithmBTest {
    Algorithm algo = new AlgorithmB()
    
    @Test
    void t1() { 
        algo.process(d')
    }

    @Test
    void t2() { 
        algo.process(d'')
    }
}

AFTER

abstract class BaseTest {
    Algorithm algo
    
    @ParameterizedTest
    @MethodSource("data")
    void sharedTests(InputType input) {
        Result result = algo.process(input)
        assertBehavior(result)
    }
    
    static Stream<Arguments> data() {
        return Stream.of(
            Arguments.of(d'),
            Arguments.of(d'')
        )
    }
}

class AlgorithmATest extends BaseTest {
    @BeforeEach
    void setup() { algo = new AlgorithmA() }

}

class AlgorithmBTest extends BaseTest {
    @BeforeEach
    void setup() { algo = new AlgorithmB() }
}

Instances

Sources

Mining DatasetsMonitoringStackOverflowLiterature (Kim et al., 2021; Deursen et al., 2001)

Dependency-free Test Parameterization

Test Method Refactorings

Motivation

The logic of many test methods may share similarities, i.e., test code duplication. When test cases differ in the Arrange phase (setup), developers may choose to parameterize their fixture or utility methods. However, support is still incipient in most test frameworks (see junit5#878). As alternatives, developers leverage method and superclass extractions to achieve the same behaviour.

Qualities

Reusability
Uniformity

Code Demonstration

BEFORE

@Test 
void t1() {
    setUp(expression)
    actual = sut.method()
    assertEquals(expected, actual)
}

@Test 
void t2() {
    setUp(expression')
    actual' = sut.method()
    assertEquals(expected', actual')
}

AFTER

void t(expression, expected) {
    setUp(expression)
    actual = sut.method()
    assertEquals(expected, actual)
}

@Test 
void t1() {
    t(expression, expected)
}

@Test 
void t2() {
    t(expression', expected')
}

Instances

Sources

MonitoringStackOverflow

Enhance Test Report

Test Method Refactorings

Variants

1. Customize Error Message2. Parameterize Test Name3. Add Assertion Explanation4. Change Order of Assertion’s Parameters5. Rename Test6. Specialize Expected Exception7. Rename Data Provider

Motivation

1. Good assertions' and exceptions' error messages are useful for debugging a failing test case.

2. Test frameworks rely on method names to identify tests in a test report; conversely, parameterized tests may benefit from custom names to distinguish the execution report of each one of its parameters.

3. Having more than one assertion with no explanation in a test method is known as the Assertion Roulette test smell. The name of tests suffering with that smell may not explain well the failing assertion.

4. Testers may misplace the expected and obtained value parameters of assertions due to conflicting standards in assertions' parameters order among Java test libraries, which incurs in misleading test reports.

5. The name of test methods and classes must indicate context, trigger, and behaviour of the SUT. Projects and communities may also enforce compliance to certain naming idioms. When one of those elements changes, the names should be updated to reflect it.

6. Incorrect use of built-in exception types may introduce ambiguity. Testers disambiguate exception testing by extracting a subclass of the ambiguous exception.

7. Like test methods, data providers should be named after the data they provide and their intended purpose. Despite different syntax, all major Java test frameworks offer a powerful name parameter for data provider annotation, in which developers may describe it in human readable format.

Qualities

Effectiveness
Uniformity

Code Demonstration

BEFORE

public class AccountService {
    public void withdraw(double amount) {
        if (amount <= 0) {
            throw new IllegalArgumentException("Invalid amount")
        }
    }
}

AFTER

public class AccountService {
    public void withdraw(double amount) {
        if (amount <= 0) {
            throw new IllegalArgumentException(
                String.format("Withdrawal amount must be positive. Got: %.2f", amount)
            )
        }
    }
}

BEFORE

@ParameterizedTest
@ValueSource()
void t(params) {  }

AFTER

@ParameterizedTest(name = "Descriptive name {arguments}")
@ValueSource()
void t(params) { }

BEFORE

assertTrue(condition)
assertNotNull(obj)

AFTER

assertTrue("Condition should hold", condition)
assertNotNull("Object must be initialized", obj)

BEFORE

assertEquals(actual, expected)

AFTER

assertEquals(expected, actual)

BEFORE

@Test
void test1() { }

AFTER

@Test
void descriptiveTestName() { }

BEFORE

@Test(expected = GeneralException.class)
void t() { }

AFTER

@Test(expected = SpecificException.class)
void t() { }

BEFORE

@DataProvider(name = "The data provider")
static Object[] source() { }

@Test(dataProvider = "The data provider")
void t(params) { }

AFTER

@DataProvider(name = "Positive cases data provider")
static Object[] source() { }

@Test(dataProvider = "Positive cases data provider")
void t(params) { }

Instances

Sources

Mining DatasetsMonitoringStackOverflowLiterature (Deursen et al., 2001; Martins et al., 2023)

Integration Test to Unit Test

Test Method Refactorings

Motivation

There are inherent trade-offs between integration testing and unit testing (Aniche, 2022; Osherove, 2009; Vocke, 2018).

Unit tests are perfect for high-churn logic and test-driven development because they are excellent at validating isolated components using mocked dependencies, offer quick execution, accurate failure localization, and require little maintenance.
Their limited scope, however, runs the risk of ignoring systemic defects brought on by component interactions.
On the other hand, integration tests confirm the interoperability of external systems and end-to-end workflows, revealing emergent problems like dataflow errors or protocol mismatches. 
However, this comes at the expense of slower execution, higher infrastructure requirements, and increased flakiness because of external dependencies.
Integration tests reduce the risk of architectural misalignment, but they come with a higher operational complexity and delayed debugging than unit tests, which place more emphasis on agility and detailed feedback.
The testing pyramid model (Vocke, 2018) is in line with a balanced strategy that prioritizes unit tests for core logic and saves integration tests for crucial interfaces.
Breaking integration tests into unit tests consists in promoting isolation through mocking and proper set-up/clean-up between consecutive test executions.
This ensures robust coverage of isolated behaviours while balancing risk mitigation with quick feedback.

Qualities

Brittleness
Independence
Efficiency

Code Demonstration

BEFORE

setupDatabaseConnection()
User user = userService.create("test@example.com")
teardownDatabaseConnection()
assertTrue(userExistsInDB("test@example.com"))

AFTER

InMemoryUserRepository fakeRepository = new InMemoryUserRepository()
UserService service = new UserService(fakeRepository)    
User user = service.create("test@example.com")    
assertTrue(fakeRepository.contains("test@example.com"))

Instances

Sources

StackOverflow

Replace Duplicate Assert with Repeat Tests

Test Method Refactorings

Motivation

When a test repeatedly evaluates the same assertion for a fixed number of times, the code can be simplified with @RepeatedTest.

Qualities

Simplicity
Changeability

Code Demonstration

BEFORE

@Test
void t() {
    stmt // 1ˢᵗ repetition
    assertEquals(expected, actual)
    ...
    stmt // mᵗʰ repetition    
    assertEquals(expected, actual)
}

AFTER

@RepeatedTest(m)
void t() {
    stmt
    assertEquals(expected, actual)
}

Instances

No instances recorded.

Sources

Literature (Kim et al., 2021)

Reuse Test Methods

Test Method Refactorings

Variants

1. Between Unit and Integration2. Across Projects

Motivation

1. Oftentimes, unit tests simply differ to integration test due to mocked dependencies; one may find useful to generalize the test code so real instances and mocks can be used interchangeably.

2. Including third-party test suites in a custom TestSuite facilitates seamless integration and execution of external test cases when using third-party plug-in components.

Qualities

Reusability
Reliability
Independence

Code Demonstration

BEFORE

@Test  
void unitT() {  
    Dependency mock = mock(Dependency.class)  
    SUT sut = new SUT(mock)  
    stmt(sut)  
    verify(mock).expectedCall()  
}

@Test  
void integrationT() {  
    Dependency real = new Dependency()  
    SUT sut = new SUT(real)  
    assertEqual(expected, stmt(sut))  
}

AFTER

abstract class BaseTest {  
    abstract Dependency dependency() 

    @Test  
    void t() {  
        SUT sut = new SUT(dependency())  
        stmt(sut)  
        assertOrVerify(expected)
    }  
}  

class UnitC extends BaseTest {  
    Dependency dependency() { 
        return mock(Dependency.class) 
    }  
}  

class IntegrationC extends BaseTest {  
    Dependency dependency() { 
        return new Dependency() 
    }  
}

Instances

Sources

StackOverflow

Introduce Test Parameter Object

Test Method Refactorings

Motivation

Like production code methods, parameterized tests with many parameters may become hard to read.

Introducing parameter objects (Fowler et al., 2012) encapsulates all parameters into one object.

Qualities

Uniformity

Code Demonstration

BEFORE

@ParameterizedTest  
@CsvSource({
    P₁₁, ..., P₁ₙ
    ...
    Pₘ₁, ..., Pₘₙ
})    
void t(P₁, ..., Pₙ) {  }

AFTER

@ParameterizedTest  
@MethodSource("data")  
void t(ParamObj P) {  }  

public static StreamArguments data() {  
    return Stream.of(  
        Arguments.of(new ParamObj(P₁₁, ..., P₁ₙ)),
        ...
        Arguments.of(new ParamObj(Pₘ₁, ..., Pₘₙ))
    )  
}

Instances

Sources

StackOverflow

Extract Utility Method

Test Method Refactorings

Variants

1. Extract Act Method2. Extract Assertion Method3. Extract Fixture Utility Method4. Delegate Assertion of Disjoint SUTs with same API

Motivation

After identifying common code that can be shared between two or more test classes, utility methods may be extracted.

1. Standardize a complex MUT (Method-Under-Test) invocation throughout the test code.

2. Combines many assertion invocations into a custom higher-level assertion method with semantically accurate name useful failure messages.

3. Encapsulates environment configuration and clean up, through data creation, deletion, and transformation without depending on test framework features and conventions, which promote higher flexibility and reusability at the cost of conciseness.

4. SUTs that share public methods with the same signatures but no common interface can hinder reusability within tests. Testers may choose to extract the assertions into an utility method that expects a lambda to invoke the MUT (Method-Under-Test).

Qualities

Reliability
Uniformity
Changeability
Reusability

Code Demonstration

BEFORE

@Test
void t1() {
    stmt
    actual = sutA.method()  
    assertEquals(expected, actual)  
    stmt'
}

@Test
void t2() {
    stmt
    actual = sutA.method()  
    assertEquals(expected, actual)  
    stmt'
}

@Test
void t3() {
    actual = sutA.method()  
    assertEquals(expected, actual) 
    actual = sutB.method()  
    assertEquals(expected, actual)  
}

AFTER

// Variant (1)
void setUp() {
    stmt
}
void tearDown() {
    stmt'
}
// Variant (2)
void act() {
    actual = sutA.method()
}
// Variant (3)
void customAssert() {
    assertEquals(expected, actual)
}

@Test
void t1() {
    setUp()
    act()
    customAssert() 
    tearDown()
}
@Test
void t2() {
    setUp()
    act()
    customAssert() 
    tearDown()
}

// Variant (4)
@Test
void TSutA() {
    testMutBehavior(() -> sutA().method())
}
@Test
void TSutB() {
    testMutBehavior(() -> sutB().method())
}
void testMutBehavior(Fuc<?> func) {
    actual = func()
    assertEquals(expected, actual) 
}

Instances

No instances recorded.

Sources

Mining DatasetsMonitoringStackOverflowLiterature (Meszaros, 2007)

Categorize Test Method

Test Method Refactorings

Motivation

Test suites may have test methods with common features (execution time, code size).

Developers may want to execute those groups separately and in a specific order.

Qualities

Efficiency

Code Demonstration

BEFORE

class C {
  @Test
  void fastTest() {}
  @Test
  void slowTest() {}
}

AFTER

@Tag("Fast")
@Test
void fastTest() {}

@Tag("Slow")
@Test
void slowTest() {}

Instances

Sources

StackOverflowGitHub

Extract Utility Class

Test Method Refactorings

Variants

1. Extract Class2. Pull-up Method

Motivation

Utility methods can be extracted and collected into a General Utility Class.

Using named Expectations in JMockit allows setting up custom shared mocks in a base class.

Qualities

Uniformity
Independence

Code Demonstration

BEFORE

class C1 {
  private void common() {}
}

AFTER

class Utils {
  public void common() {}
}
class C1 {
  @Test void t() { Utils.common() }
}

Instances

Sources

StackOverflowGitHub

Split Test Method

Test Method Refactorings

Motivation

Test methods may evolve and accumulate multiple responsibilities (Eager Test).

Test methods can be separated in multiple pure test methods to improve the execution trace for debugging.

Qualities

Uniformity

Code Demonstration

BEFORE

@Test
void t() {
  assert(A)
  assert(B)
}

AFTER

@Test
void t1() { assert(A) }

@Test
void t2() { assert(B) }

Instances

Sources

StackOverflow

Merge Test Method

Test Method Refactorings

Motivation

Test suites can accumulate redundancies over time.

When such redundancy is scattered throughout multiple test methods, we can eliminate code duplication by merging them.

Qualities

Changeability
Efficiency
Simplicity

Code Demonstration

BEFORE

@Test
void t1() { assert(A) }
@Test
void t2() { assert(B) }

AFTER

@Test
void t() {
  assert(A)
  assert(B)
}

Instances

SO #5666415

Sources

StackOverflow

Migrate Categories

Migration Refactorings

Motivation

TestNG's Groups, JUnit 4's Category, and JUnit 5's Tags are equivalent features but differ in flexibility.

Tags (JUnit 5) integrate with other features such as custom display name.

Qualities

Simplicity
Effectiveness

Code Demonstration

BEFORE

@Test
@Category(FastTests.class)
void t() {}

AFTER

@Tag("fast")
@Test
void t() {}

Instances

Sources

GitHub

Migrate Runner

Migration Refactorings

Motivation

Migrating tests with custom runners may require adapting JUnit 4’s Runner API to JUnit 5’s Extension API.

Qualities

Reliability

Code Demonstration

BEFORE

@RunWith(SpringJUnit4ClassRunner.class)
class SpringTest {}

AFTER

@ExtendWith(SpringExtension.class)
class SpringTest {}

Instances

Sources

GitHub

Migrate Mock

Migration Refactorings

Variants

1. To Mockito2. To Powermock

Motivation

Ease of use and active development are reasons to migrate to Mockito.

Powermock includes support for mocking static methods where earlier frameworks might not.

Qualities

Changeability

Code Demonstration

BEFORE

// EasyMock
expect(mock.method()).andReturn(val)
replay(mock)

AFTER

// Mockito
when(mock.method()).thenReturn(val)

Instances

Sources

GitHubStackOverflow

Migrate Fixture

Migration Refactorings

Motivation

Updates annotations from JUnit 4 style (@Before, @BeforeClass) to JUnit 5 style (@BeforeEach, @BeforeAll).

Qualities

Reusability

Code Demonstration

BEFORE

@Before
void setUp() {}
@BeforeClass
static void setupClass() {}

AFTER

@BeforeEach
void setUp() {}
@BeforeAll
static void setupClass() {}

Instances

Sources

GitHub

Migrate Assertion

Migration Refactorings

Variants

1. JUnit to Hamcrest2. To AssertJ3. JUnit 4 to 5

Motivation

Adopted test library may lack desired features (fluent API).

AssertJ is a fluent assertions library compatible with most test frameworks.

JUnit 5 introduces new assertions like assertThrows.

Qualities

Uniformity

Code Demonstration

BEFORE

assertEquals(expected, actual)

AFTER

assertThat(actual).isEqualTo(expected)
// or JUnit 5
assertThrows(E.class, () -> stmt)

Instances

Sources

GitHubStackOverflow

Migrate Parameterized Test

Migration Refactorings

Variants

1. TestNG to JUnit 52. JUnit 4 to JUnit 5

Motivation

JUnit 5 supports parameterization by default, allows multiple argument providers, and includes over ten provider types (CSV, Method, etc).

Qualities

Simplicity
Reusability

Code Demonstration

BEFORE

@Test(dataProvider = "data")
void t(P) {}

AFTER

@ParameterizedTest
@CsvSource({P1, P2})
void t(P) {}

Instances

Sources

GitHub

Migrate Expected Exception

Migration Refactorings

Motivation

Using try-catch or @Test(expected=...) enables false positives.

JUnit 5's assertThrows preserves execution of remaining statements and targets specific calls.

Qualities

Simplicity

Code Demonstration

BEFORE

@Test(expected = E.class)
void t() { stmt }

AFTER

@Test
void t() {
  assertThrows(E.class, () -> stmt)
}

Instances

Sources

GitHubStackOverflow

Replace Test Annotation between Class and Method

Migration Refactorings

Motivation

TestNG’s @Test annotation can be used on classes. Migration often involves switching scope.

Qualities

Simplicity
Uniformity

Code Demonstration

BEFORE

class C {
  @Test public void t1() {}
}

AFTER

@Test
class C {
  public void t1() {}
}

Instances

Sources

GitHub

Migrate Test Timeout

Migration Refactorings

Motivation

JUnit 5 provides a separate annotation (@Timeout) rather than a parameter on @Test.

Qualities

Changeability
Uniformity

Code Demonstration

BEFORE

@Test(timeout = 5000)
void t() {}

AFTER

@Timeout(5)
@Test
void t() {}

Instances

Sources

GitHub

Replace Loop

Test Smell Refactorings

Variants

1. Replace Fixed Loop with Repeated Tests2. Replace For-each with Parameterized

Motivation

When a test repeatedly evaluates the same assertion for a fixed number of times, it can be simplified.

Testing permutation of values with nested loops should be parameterized.

Qualities

Uniformity
Changeability

Code Demonstration

BEFORE

@Test
void t() {
  for(int i=0; i<5; i++) {
    assert()
  }
}

AFTER

@RepeatedTest(5)
void t() {
  assert()
}

Instances

SO #69010737

Sources

StackOverflow

Replace Mystery Guest with @TempDir

Test Smell Refactorings

Motivation

Using external resources (files) is a symptom of Mystery Guest.

Using @TempDir ensures resources are created and cleaned up automatically.

Qualities

Independence

Code Demonstration

BEFORE

@Test
void t() {
  File.createTempFile(...)
}

AFTER

@Test
void t(@TempDir File D) {
  D.createTempFile(...)
}

Instances

Commit: Flink

Sources

GitHub

Solve Race Condition with Resource Lock

Test Smell Refactorings

Motivation

Running test cases concurrently is challenging when relying on shared resources.

Using @ResourceLock synchronizes resource acquisition.

Qualities

Independence

Code Demonstration

BEFORE

@Execution(CONCURRENT)
class C { ... }

AFTER

@Execution(CONCURRENT)
class C {
  @ResourceLock(value=SYS_PROPS)
  void t() {}
}

Instances

Sources

StackOverflow

Inline Resource

Test Smell Refactorings

Motivation

Tests that use external resources are not self-contained.

Incorporate the resource content into the test code to remove dependency.

Qualities

Uniformity
Changeability
Effectiveness

Code Demonstration

BEFORE

res = loadResource("file.txt")

AFTER

value = """
  INLINED_DATA
"""

Instances

No instances recorded.

Sources

Literature

Setup External Resource

Test Smell Refactorings

Motivation

Optimistic assumptions about external resources cause non-deterministic behavior.

Explicitly create and release resources before/after testing.

Qualities

Reliability

Code Demonstration

BEFORE

void t() {
  process("data.csv")
}

AFTER

@BeforeEach
void setUp() { create("data.csv") }
@AfterEach
void tearDown() { delete("data.csv") }

Instances

No instances recorded.

Sources

Literature

Make Resource Unique

Test Smell Refactorings

Motivation

Overlapping resource names cause crashes in concurrent runs.

Use unique identifiers (e.g., timestamps) for allocated resources.

Qualities

Reliability

Code Demonstration

BEFORE

createResource("shared.txt")

AFTER

name = "res-" + timestamp
createResource(name)

Instances

No instances recorded.

Sources

Literature

Introduce Equality Method

Test Smell Refactorings

Motivation

Test methods with multiple assertions on object properties are hard to debug.

Compare objects with an expected object (equals method) rather than properties separately.

Qualities

Changeability
Uniformity
Brittleness
Simplicity

Code Demonstration

BEFORE

assertEquals(e1, act.p1)
assertEquals(e2, act.p2)

AFTER

assertEquals(expectedObj, actualObj)

Instances

Sources

StackOverflow

Mock Time Input

Mocking Refactorings

Motivation

Testing software relying on clock time is challenging.

Mock the time to speed-up tests and avoid synchronization problems.

Qualities

Efficiency
Effectiveness

Code Demonstration

BEFORE

Thread.sleep(5000)

AFTER

Clock mockClock = Clock.fixed(...)
cache.setClock(mockClock)

Instances

Sources

StackOverflow

Replace Mocks with Real Instances

Mocking Refactorings

Motivation

Mocks might behave differently than real objects.

Simpler dependencies may not benefit from mocking.

Replacing mock with real instance can be beneficial if performance is satisfactory.

Qualities

Uniformity

Code Demonstration

BEFORE

when(mock.method()).thenReturn(val)
sut = new SUT(mock)

AFTER

real = new RealDep()
sut = new SUT(real)

Instances

Sources

StackOverflowGitHub

Replace Real Instances with Mocks

Mocking Refactorings

Motivation

Mocks are efficient and isolate components.

Useful to prevent undesired access to databases and external APIs.

Qualities

Independence
Efficiency
Reliability

Code Demonstration

BEFORE

db = new CloudDB()
sut = new SUT(db)

AFTER

mockDb = mock(CloudDB.class)
sut = new SUT(mockDb)

Instances

Sources

StackOverflowGitHub

Reuse Mock

Mocking Refactorings

Motivation

When specific features are heavily used, many tests require mocking it.

Extract a Fake or Stub class used across many tests.

Qualities

Reusability
Changeability

Code Demonstration

BEFORE

mock = mock(Dep.class)
when(mock.m()).thenReturn(v)

AFTER

class FakeDep extends Dep { ... }
dep = new FakeDep()

Instances

Sources

StackOverflow

Replace Inheritance by Mocking API

Mocking Refactorings

Motivation

Inheritance requires manually crafting tracking logic.

Mocking APIs provide verification mechanisms out of the box.

Qualities

Simplicity

Code Demonstration

BEFORE

class T extends MockBase {
  void t() {
    assertCustomTracking(dep)
  }
}

AFTER

@Mock Dep dep
void t() {
  verify(dep).method()
}

Instances

No instances recorded.

Sources

Literature

Replace Mocking API with Anonymous Subclass

Mocking Refactorings

Motivation

Java's anonymous classes are commonly used for creating stubs.

Wrapping them with Spies can be redundant; manual implementation may be simpler.

Qualities

Reusability

Code Demonstration

BEFORE

spy(new Dep() { ... })

AFTER

new Dep() { ... }

Instances

Commit: JDBI

Sources

GitHub

Replace Test Double Type

Mocking Refactorings

Motivation

Switching between Dummy, Stub, Fake, Spy, and Mock based on needs (flexibility vs simplicity).

E.g., replacing a Stub with a Mock for verification capabilities.

Qualities

Efficiency
Simplicity

Code Demonstration

BEFORE

Dependency stub = new Dependency() { ... }

AFTER

when(mock.method()).thenReturn(val)

Instances

Commit: Novoda

Sources

GitHub

Replace Test Double with ObjectMother

Mocking Refactorings

Motivation

Centralize complex mock creation logic.

Object Mother provides specific versions of complex type instances.

Qualities

Uniformity
Efficiency

Code Demonstration

BEFORE

mock = mock(Dep.class)
when(mock.m()).thenReturn(val)

AFTER

sut = new SUT(
  ObjectMother.createConfiguredDep()
)

Instances

Commit: Novoda

Sources

GitHub

Standardize Test Utilities for Project-Wide Reuse

Test Suite Refactorings

Variants

1. Generalize Utility Class2. Export as Framework3. Reuse Fixtures4. Reuse Cached Context5. Reuse Cache DB

Motivation

Generalize utility methods' return types/parameters to promote reuse.

Export utility classes to share logic among projects.

Reuse cached Spring contexts to avoid initialization overhead.

Qualities

Reusability
Efficiency

Code Demonstration

BEFORE

void t() {
  ProjectUtils.specificHelper(p)
}

AFTER

void t() {
  Framework.generalHelper(p)
}

Instances

Sources

StackOverflowGitHub

Custom Runner

Test Suite Refactorings

Motivation

Use JUnit 4 rules or subclass existing runners to customize behavior.

Useful when only one runner is allowed per class.

Qualities

Reusability
Changeability

Code Demonstration

BEFORE

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(...)

AFTER

@RunWith(CustomRunner.class)

Instances

Sources

StackOverflowGitHub

Custom Annotation

Test Suite Refactorings

Motivation

Custom annotations tag tests for integration or attach metadata.

Composes multiple annotations (Tag, Timeout, Test) into one.

Qualities

Uniformity

Code Demonstration

BEFORE

@Test
@Timeout(5)
@Tag("integration")
void t() {}

AFTER

@IntegrationTest
void t() {}

Instances

Sources

StackOverflowGitHub

Restructure Test Suite

Test Suite Refactorings

Variants

1. Nest Classes2. Move Method3. Denest Classes

Motivation

Nested test classes organize tests into logical groups with scoped fixtures.

Moving test methods reflects changes in production code structure.

Qualities

Uniformity

Code Demonstration

BEFORE

class FlatTest {
  @Test void t1() {}
  @Test void t2() {}
}

AFTER

class Outer {
  @Nested class C1 { @Test void t1() {} }
  @Nested class C2 { @Test void t2() {} }
}

Instances

Sources

StackOverflowGitHub

Extend Framework

Test Suite Refactorings

Variants

1. Test Extension2. Enforce Architecture

Motivation

JUnit 5 extension model prioritizes composition.

ArchUnit allows teams to define architectural rules in Java.

Qualities

Uniformity
Reusability
Changeability

Code Demonstration

BEFORE

void t() {
  try { ... } finally { release() }
}

AFTER

@ExtendWith(ResourceExt.class)
void t(Resource r) { ... }

Instances

Sources

StackOverflowGitHub

Catalogue Browser

Lazily Evaluate Values

Motivation

Qualities

Code Demonstration

Instances

Sources

Consolidate Multiple Assertions into a Fluent Assertion

Motivation

Qualities

Code Demonstration

Instances

Sources

Group Multiple Assertions

Motivation

Qualities

Code Demonstration

Instances

Sources

Replace Assertion

Motivation

Qualities

Code Demonstration

Instances

Sources

Enforce Use of Type Inference

Motivation

Qualities

Code Demonstration

Instances

Sources

Abstract away SUT's Construction

Motivation

Qualities

Code Demonstration

Instances

Sources

Extract Preconditions

Motivation

Qualities

Code Demonstration

Instances

Sources

Reuse code with Fixture / Extract Fixture

Motivation

Qualities

Code Demonstration

Instances

Sources

Inline Fixture

Motivation

Qualities

Code Demonstration

Instances

Sources

Minimize Fixture

Motivation

Qualities

Code Demonstration

Instances

Sources

Override Fixtures

Motivation

Qualities

Code Demonstration

Instances

Sources

Inject SUT Dependency

Motivation

Qualities

Code Demonstration

Instances

Sources

Replace Class Fixture with Method Fixture

Motivation

Qualities

Code Demonstration

Instances

Sources

Replace Method Fixture with Class Fixture