Mock frameworks

Mocking frameworks are quite popular in java, and have matured a bit in recent years. Now is a good time to take a look at the top frameworks, and decide which one is best in certain circumstances, and to see why older or home-brewed variants aren’t as solid as these frameworks. Since I think I built up a slightly alternative view on the subject, I felt obliged to share it with the world. So I’ll try to give you my perspective on what I will (try to) use.

NOTE: This article is outdated. Check Mockito vs JMockit for a modern view.

So I will start with giving some background information about mock frameworks.

Different approaches

There are 2 approaches for mock frameworks. It can either be proxy-based or be based on bytecode manipulation.
Proxy-based means the framework allows you to create mocks that look like the object you’re mocking, but aren’t of that class. For interfaces this just means returning an implementation of that interface. It’s more complicated for classes. The framework will basically try to subclass the given class and overwrite its methods with some function that registers and returns the given content.

But this approach has its downsides, you can not mock a final class, or static or final methods. And if you want to return a mock when a constructor is called, it won’t be possible. 
Most frameworks including Mockito and EasyMock use this approach.

The other possibility is to use bytecode manipulation. Which means an object will be created and at runtime its inner workings will be manipulated to allow the usual mocking behaviour (like the metadata in groovy allows you to add / alter methods). This does not change much for interfaces, but for objects it resolves the downsides the proxy-based mock frameworks have. This can also be done for classes, which allows the mocking of static methods. But the downside is that if you’d initiate an object with that class in the mean time, it would be altered too. So you have to understand the concept behind it when you use it.

JMockIt and Powermock (basically an extension of both EasyMock and Mockito) use this approach.

The advantages of mocking

The obvious advantage of mocking is that you don’t have to write any mocks/stubs yourself. Mocks allow you to disconnect units in your code, thereby allowing unit- and some kinds of integration-tests.
The underlying thought of testing is that it functions as a contract, and you have a way to notice when the contract is broken or improve the contract when needed. One of the mistakes that can be made here is to make a contract too stringent, and especially with mock objects you can do this easily.
The remedy is to do TDD correctly, so you start writing your test first. And afterwards (and after altering a test), check whether there are implementations that would work just as well but fail on the current test.

Mocking itself also became a sanity check for the code. If a certain method isn’t easy to test, maybe the code itself isn’t flexible enough. Usually this boils down to known smells.
Like having to use too many mock objects in a test method means there is high coupling, too many or too complicated tests means the code itself should be split up etc.

To have a need for something that involves constructor mocking or static method mocking (e.g. what is impossible with proxy-based frameworks) tends to be seen as something that should only arise when you’re testing legacy code.

I have taken a look at the 3 most popular frameworks for mocking: Mockito, EasyMock and jMockIt. The first 2 are proxy-based, the third uses bytecode enhancement. This makes comparing the first 2 relatively easy, but comparing those to jMockIt is a step too far for one blog post. So in the next blog post I will compare jMockIt agains Mockito with PowerMock, which makes them more comparable. And in the remainder of this post I will compare EasyMock and Mockito.

EasyMock vs. Mockito

I will begin with a simple code example in both frameworks:

Mockito:

Request requestMock = mock(Request.class);
when(requestMock.getMethod()).thenReturn(“POST”);
testMethod(requestMock);
verify(requestMock, times(2)).getMethod();

EasyMock:

Request requestMock = createNiceMock(Request.class);
expect(requestMock.getMethod()).andReturn(“POST”).times(2);
replay(requestMock);
testMethod(requestMock);
verify(requestMock);

As you can see, the structure is quite similar between the two framework. Mockito still uses bits of the codebase of EasyMock actually. But one important difference can already be seen here. Mockito makes a clear difference between defining the results of the mocked methods (before) and the verification whether they were called (after). EasyMock does both things before actually testing the method. This has some consequences, like the collect-construction in EasyMock that actually makes it possible to verify after the method, and Mockito not failing directly when a method was called too many times (in the example, it would return “POST” 5 times if necessary).  These are relatively small differences, the possibilities of both frameworks are about equal and the methods do resemble each other quite a lot. And they have the same drawbacks, since they are both proxy-based.

That really shows in this article , from the lead developer of EasyMock. It shows the similarities between the two frameworks, and the fact that EasyMock was actually following Mockito a bit when upgrading to version 3.

This really shows when you’re looking for documentation on EasyMock, most of the links you find still mention the now obsolete ‘class extensions’, and it shows that the popularity of EasyMock is waning.

When comparing the official documentation of EasyMock and Mockito, something else caught my eye. EasyMock is just basic documentation: a is possible, b is possible, and some examples. Mockito’s documentation contains things like “WARNING: This feature should rarely be required for regular clean code! ...”, and states that the need for partial mocks should be a rare exception, and answers like these . This shows a real focus on creating clean test code, and since test code should be maintainable and readable, this makes a lot of sense. And this comes up consistently in the documentation and blog posts by the makers. This resulted in a clear API which really allows clean tests.

Conclusion

Mockito’s focus on clean code and the fact that EasyMock has a bit more legacy, made Mockito more popular than EasyMock. The difference is not really in the possibilities (even though Mockito does try new things, like spying and incorporating hamcrest matchers), but in the way they see testing. And that strikes me as the right way to look at it, which makes it an easy victory for Mockito.