JMockit - Test randomly fails or succeeds

tl;dr

Tests randomly fail or succeed. Don't like that.

Code

The method under test:

public void prepareCopy(Long selectedEntity) throws EntityNotFoundException {
        System.out.println(selectedEntity); // for debugging this problem
        T template = prepareEditableObject(selectedEntity);
        System.out.println(template); // for debugging this problem
        this.technicalObject = prepareCopiedObject(template);
        this.viewType = ViewType.CREATE;
        reloadReferencedObjects();
        this.viewState = ViewState.DISPLAY;
        performAction(ViewEvent.SHOW, ViewType.CREATE);
    }

The test:

@RunWith(JMockit.class)
public class EditorPMUnitTest {

@Tested
// this class inherits the method preparyCopy(Long selectedEntity)
BackOfficeEditorPM sut;

@Injectable 
// for all dependencies

@Test
public void testPrepareCopy() throws Exception {
    /* PREPARATION */
    // there is no randomness in this method apart a LocalDateTime validFrom = LocalDateTime.now();
    BackOffice dummyBackoffice = TestUtils.createBackOfficeDummy();

    /* RECORD */
    // we also tested this passing only sut, not dummyBackoffice --> same problem
    new Expectations(sut, dummyBackoffice) {
        {
            // also tried 1L and withInstanceLike(1L)
            invoke(sut, "prepareEditableObject", anyLong);
            result = dummyBackoffice;
            invoke(sut, "performAction", ViewEvent.SHOW, ViewType.CREATE);
            invoke(sut, "reloadReferencedObjects");
        }
    };

    /* REPLAY */
    // dummyid because the usage of this id is mocked in the prepareEditableObject method
    final Long dummyId = 1L;
    // also tried with just passing 1L
    sut.prepareCopy(dummyId);

    /* VERIFY */

    // get to check variables in PM
    ViewType edit = sut.getViewType();
    BackOffice backoffice = sut.getBackOffice();
    boolean confirmed = sut.isConfirmation();

    new Verifications() {
        {
            // we changed anyLong accordingly to the changes in Expectations
            invoke(sut, "prepareEditableObject", anyLong);
            times = 1;

            invoke(sut, "reloadReferencedObjects");
            times = 1;

            invoke(sut, "performAction", ViewEvent.SHOW, ViewType.CREATE);
            times = 1;

        }
    };

    assertNull(backoffice.getId());
    assertEquals(ViewType.CREATE, edit);
    assertEquals(backoffice.getBezeichnung(),
            dummyBackoffice.getBezeichnung());
    assertFalse(confirmed);
}

}

As you can see all methods (except the one below) are mocked away.

@Override
protected BackOffice prepareCopiedObject(BackOffice template) {
    BackOffice newBO = template.cloneInstance();
    newBO.setNummer(null);
    return newBO;
}

The method cloneInstance creates a new object from the fields of the object the method being called from. There is no logic.

Problem

The tests fail or succeed randomly. The results of multiple runs (in isolation of other tests) were:

• success
• fail
• fail
• success
• fail
• success
• fail
• fail
• success
• success
• success
• success
• success
• success
• fail
• success

Much randomness indeed.

This is the output when the test succeeds (originates from the syso debugging above):

1
my.package.backoffice.BackOffice@1

This is the error thrown when a test fails:

java.lang.IllegalStateException: Missing invocation to mocked type at this point; please make sure such invocations appear only after the declaration of a suitable mock field or parameter
    at my.package.otherpackage.EditorPMUnitTest$24.<init>(EditorPMUnitTest.java:646)
    at my.package.otherpackage.EditorPMUnitTest.testPrepareCopy(EditorPMUnitTest.java:643)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.lang.reflect.Method.invoke(Method.java:483)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:234)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:74)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.lang.reflect.Method.invoke(Method.java:483)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)

This error basically says that the expectation invoke(sut, "prepareEditableObject", anyLong); was not fulfilled, even though the test was not yet able to actually call the method as shown in the next section.

Test execution breaks apart

With the above it looks like sometimes sut.prepareCopy(dummyId) is called correctly and sometimes not, which again leads me to the assumption that the JMockit initializer is not working properly.

Next try was to debug through the code and see where it fails in which situation.

Failing tests reach step 1 and 2, but not 3 or 4. Note that we also tried anyLong and withInstanceLike(1L) as parameters at breakpoint 2.

Question

Why does JMockit sometimes break apart at that line? How can we fix this issue so that the test doesn't fail randomly?

---

System details:

• Windows 7
• JMockit 1.15
• JUnit 4.11
• JDK 1.8
• Eclipse/IntelliJ IDEA 15.0.4/5