Mockito / PowerMock vs JMockit

If you are on the verge of choosing between Mockito and JMockit, this article will help you understand why you should pick JMockit. When aiming for a Java codebase which is well-covered with unit tests, you are certain to run into mocking frameworks. The two most popular ones are Mockito and JMockit. Mockito, the successor to the older EasyMock, teams up with PowerMock to have complete functionality. JMockit, an ambitious young library aiming for nothing less than taking Mockito's place, works all by itself.

Introducing the frameworks


In my previous blog post on mocking, I looked mainly at Mockito. Now, I've looked at JMockit as well.

First off, JMockit looks a bit weird. You have to get your head around its funny syntax, but once you do it feels quite natural.
new Expectations() {
SomeObject object {
object.doSomething();
result = "hello world";
}};

Note the double curly brackets. What you are doing here is creating an anonymous inner class with an instance initialization block. The object after the first curly bracket is automatically mocked. The logic after the second curly bracket becomes your expectation logic. You call a method on a mock and then you decided what to do with. So, in this example you have a mock of type SomeObject. In the initialization block, the method doSomething on SomeObject is instructed to return "hello world".

To compare, in Mockito, you would write:
SomeObject object = mock(SomeObject.class);
when(object.doSomething()).thenReturn("hello world");

Seeing this, JMockit certainly starts off on the back foot. However, Mockito's apparent advantage starts to crumble when you mock void methods, as you suddenly have to invert the order:
doThrow(new Exception()).when(someObject).returnsVoid();

You can either love or hate the syntax that both frameworks employ, however, what matters most is how both frameworks deal with the hard-to-mock parts.

Mocking aspects


The following aspects I found important:

  • mocking new; mocking the new constructor

  • mocking static methods; mocking static methods on classes

  • partially mocking instances; having a real instance and mocking a subset of methods

  • partially mocking enums; mocking a subset of methods on enum instances

  • capturing arguments; being able to capture arguments and asserting those

  • fluent interfaces; having no trouble using fluent interfaces (ie, methods that return this)

  • consistency of mocking library; besides the aspects mentioned before, it is important that a library has a consistent interface


During the last couple of days, I migrated the entire JOSS codebase away from Mockito & Powermock to JMockit. This puts me in the perfect spot to appraise those aspects for both sides.

If you are already making use of Mockito / PowerMock and consider migrating to JMockit, it might be interesting to know that roughly 20% of my time was needed to move away from PowerMock, whereas 80% was eaten up by Mockito. If you asked me up front, I would have guessed the opposite, since most of my time with Mockito was spent wrestling with the harder parts (ie, PowerMock). Migrating to JMockit is not a matter of just switching some libraries. The different syntax of both libraries makes a migration non-trivial.

Mocking new


Mocking the new constructor is functionality which you get for free in JMockit. As soon as you declare a method @Mocked, that is the instance you will be talking to when you instantiate a new instance. However, it is possible to declare the new instance in the expectations and return your own mocked instance.
@Test
public void doSomething(@Mocked(stubOutClassInitialization = false) final ToDependency toDep) throws IOException {
ServiceWithNewDependency service = new ServiceWithNewDependency();
new Expectations() {{
new ToDependency();
result = toDep;
toDep.crashHorribly();
result = 5;
}};
assertEquals(9, service.doSomething());
}

In Mockito, you will have to employ whenNew, in combination with preparing the class for its use. The difference between the two is great and this is a sure win for JMockit.
@RunWith(PowerMockRunner.class)                 
@PrepareForTest(ServiceWithNewDependency.class)
public class ServiceTest {

@Test
public void doSomething() throws Exception {
ToDependency toDep = mock(ToDependency.class);
when(toDep.crashHorribly()).thenReturn(5);
whenNew(ToDependency.class).withNoArguments().thenReturn(toDep);
ServiceWithNewDependency service = new ServiceWithNewDependency();
assertEquals(9, service.doSomething());
}
}

Mocking static methods


Again, a breeze in JMockit. You just declare the class you want to mock and call its method in the Expectations block.
@Test
public void doSomething(@Mocked final ToDependency toDep) throws IOException {
ServiceWithStaticDependency service = new ServiceWithStaticDependency();
new Expectations() {{
ToDependency.crashHorribly();
result = 5;
}};
assertEquals(9, service.doSomething());
}

In Mockito, you have to use mockStatic and prepare the class for it, which, strangely enough, works differently than the class preparation for whenNew.
@RunWith(PowerMockRunner.class)
@PrepareForTest(ToDependency.class)
public class ServiceTest {

@Test
public void doSomething() throws Exception {
mockStatic(ToDependency.class); // @PrepareForTest is not enough, you once more prepare it
when(ToDependency.crashHorribly()).thenReturn(5);
ServiceWithStaticDependency service = new ServiceWithStaticDependency();
assertEquals(9, service.doSomething());
}
}

Partially mocking instances


Let's say you have an instance and you only want certain methods to be mocked, but leave the rest untouched. Once more, easy to do in JMockit. You pass the instance to the constructor of Expectations and you modify what you need to modify.
@Test
public void doSomething() throws IOException {
final ServiceWithPrivateDependency service = new ServiceWithPrivateDependency();
new Expectations(service) {{
invoke(service, "crashHorribly");
result = 5;
}};
assertEquals(9, service.doSomething());
}

Note that in this case the method is private. This is a time when JMockit leaves its consistent interface, simply because it cannot see private methods. Mockito does this using the spy() mechanism. The syntax is similar to what JMockit does here. Both libraries advice against mocking private methods. Note that I had some trouble getting this to work for Mockito in the test project.

Partially mocking enums


Not so different from the previous item, enum instances can be passed to JMockit in the same way as regular instances. They are modified only for those methods that need to be mocked.
@Test
public void doSomething(@Mocked final ToDependency toDep) throws IOException {
ServiceWithEnumDependency service = new ServiceWithEnumDependency();
new Expectations() {{
ToDependency.TO_DEPENDENCY.crashHorribly();
result = 5;
}};
assertEquals(9, service.doSomething());
}

For Mockito it works in the same way as the partially mocked instances from the previous chapter. In setting this up, I had some troubles to get it working.

Capturing arguments


JMockit allows you to read the arguments from the called methods in the Verifications block. However, I found withCapture not to be working for me. Instead, I solved it with forEachInvocation, which did the trick. In a conversation with Rogerio (author of JMockit), he pointed out an error in my test and also said there is a much better way to test this, using Expectations in combination with the times variable, instead of Verifications.

Mockito also allows you to capture arguments. Mockito's way is less clear, more so if withCapture works fine. JMockit allows you to assign the value directly to a variable, whereas Mockito does not.

Fluent interfaces


Since fluent interfaces are becoming more and more popular, it is nice if a mocking framework does not force you to write extra plumbing to make sure your mock returns itself after calling a fluent setter.
@Test
public void doSomething(@Cascading Fluent fluent) {
fluent.say().hello().world();
}

JMockit lets you prepend a mocked instance with @Cascading and it will automatically create the fluency code for your mock. I found this to be very handy. Mockito allows something similar, albeit in a different way.
@Test
public void doSomething() {
Fluent fluent = mock(Fluent.class, Mockito.RETURNS_DEEP_STUBS);
fluent.say().hello().world();
}

Consistency of mocking library


JMockit has a consistent interface regardless of the type of mock you work with. It is a relief to see how most bits of code look nearly alike and how easy it becomes to type the test without looking up the docs.

I found Mockito's lack of consistency to be annoying, drawing attention away from where I want it -- the test.

Waning Mockito, waxing JMockit


This is what I am annoyed by in Mockito / PowerMock:

  • confusion over what runners to use:
    @RunWith(MockitoJUnitRunner.class)
    @RunWith(PowerMockRunner.class)


  • confusion over the class to use in @PrepareClassFor for whenNew and mockStatic

  • confusion over the same names in both Mockito and PowerMock, for example the when method

  • many, many different ways to do different things, you constantly have to mentally context switch to another way of thinking

  • struggling with Mockito to get things to work


Caveats on JMockit


There are a number of items that are worth mentioning with JMockit:

  • dependency declaration; in your Maven POM, you have to make sure to declare JMockit before JUnit. If you do not, your mocks will not be created. In Mockito you would force this by using a test runner.

  • when I used a @Mocked File on a test class, Cobertura completely failed to build its coverage site.

  • when I tried to get the arguments of a verified method, withCapture() failed for me. I got things working with forEachInvocation. Rogerio pointed out that my test was not written in the spirit of the library, misusing certain aspects. Nevertheless, he fixed the issue anyway!

  • @Mocked and @Injectable are two important aspects. The first one applies to all instances of the class, whereas the second one only applies to one particular instance

  • the Expectations class forces the order and calling of all methods declared within, whereas NonStrictExpectations does not. I found myself switching between these two a lot.


Conclusion


Mockito's syntax changes depending on what you aim to accomplish. It needs an auxiliary framework (PowerMock) to offer a (near) complete set of functionality. Sadly, the two have overlapping names which is very confusing and sometimes leads to annoying errors. The way you have to prepare classes for instantiation and constructing new instances is quirky and inconsistent. Mockito / Powermock feels like a continuous struggle against the tooling, instead of writing tests.

JMockit, on the other hand, has a weird syntax to start with, but it does not take a lot of time to get used to. After that, it remains very consistent, no matter what kind of test you are pulling off. The Expectations and Verifications keep the same form that they had before. It is simply incredible to see that it is just as easy to write mocks for enums, static methods, new instances. It does not matter for JMockit. JMockit is complete and really feels like it helps to focus on writing the tests.

The biggest argument in favor of JMockit is the ease of use and how it helps insteads of hinders. Note that I have 80 hours of Mockito experience, and only 20 hours of JMockit. Despite the difference, JMockit already feels like a trusted friend. There are frameworks which cost a lot more time to properly master.

If you want to play around with the test cases, check out the project in github.