In-memory MongoDB for unit and integration tests

A few weeks ago I found myself having to fix a bug in a production system which uses MongoDB as its primary means of storage. As I was unfamiliar with the codebase, we had just taken over the project, the first thing you do is trying to find the test covering this functionality.

Jaw drop; no test in sight. What was the case, none of the interactions with the backing storage was under any form of testing. So it could happen that a simple aggregation query wasn't returning the expected results

This was my first project in which I used MongoDB, coming from projects using HSQLDB to test the validity and outcome of queries, the first thing that flashed through my mind was in-memory MongoDB. The first hit on Google wasn't promising http://stackoverflow.com/questions/10005697/does-mongo-db-have-an-in-memory-mode, but luckily some following results hit the jackpot.

Embedded MongoDB


First I started out with: Embedded MongoDB. This is actually quite neat, it acts as a bridge between java and mongo, it downloads and fires-up a real mongo instance. This has the benefit that you are talking to an instance with the same capabilities as your production environment.

Setup


At 42 we do a lot with Spring and what is easier than bootstrapping your JUnit tests using a Spring applicationcontext. To help setup Embedded MongoDB I also used https://github.com/jirutka/embedmongo-spring which is a nice builder to initialise Embedded MongoDB.
<dependency>
    <groupId>cz.jirutka.spring</groupId>
    <artifactId>embedmongo-spring</artifactId>
    <version>1.3.0</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>de.flapdoodle.embed</groupId>
    <artifactId>de.flapdoodle.embed.mongo</artifactId>
    <version>1.46.1</version>
    <scope>test</scope>
</dependency>

A Spring java configuration file could look something like:
@Configuration
public class IntegrationTestApplicationConfig extends AbstractMongoConfiguration {

    @Autowired
    private Environment env;

    @Override
    protected String getDatabaseName() {
        return env.getRequiredProperty("mongo.db.name");
    }

    @Override
    public Mongo mongo() throws Exception {
        return new EmbeddedMongoBuilder()
            .version("2.6.1")
            .bindIp("127.0.0.1")
            .port(12345)
            .build();
    }
}

But for running unit tests, as they claim, I find it a bit heavy weight. It requires a connection to the outside world to download a version of MongoDB. (Though it's also possible to set it up with a location where it can find the MongoDB packages locally). But spinning up a MongoDB seems a bit overkill. If you're running separated spring configurations for different unit tests you could even end up doing this multiple times

Fake Mongo (Fongo)


So I ended up using Fongo as it covered the basic needs of the project I was working on. It doesn't support al the functions that MongoDB offers but the basic CRUD and aggregations are supported.

Setup


Again the maven and Spring application configuration basics
<dependency>
   <groupId>com.github.fakemongo</groupId>
   <artifactId>fongo</artifactId>
   <version>1.5.8</version>
   <scope>test</scope>
</dependency>

With a Spring java configuration file looking like:
@Configuration
public class UnitTestApplicationConfig extends AbstractMongoConfiguration {
    @Autowired
    private Environment env;

    @Override
    protected String getDatabaseName() {
        return env.getRequiredProperty("mongo.db.name");
    }

    @Override
    public Mongo mongo() throws Exception {
        return new Fongo(getDatabaseName()).getMongo();
    }
}

Running the tests


I created a base class to help me run the unit tests. This class helps me import collections into the in-memory mongo instance.
@ActiveProfiles({ "test", "unit" })
@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(classes = { ApplicationConfig.class })
public abstract class SpringUnitTest {

    @Autowired
    private MongoTemplate mongoTemplate;

    protected void importJSON(String collection, String file) {
        try {
            for (Object line : FileUtils.readLines(new File(file), "utf8")) {
                mongoTemplate.save(line, collection);
            }
        } catch (IOException e) {
            throw new RuntimeException("Could not import file: " + file, e);
        }
    }
}

And finally the actual test class
public class StudentRepositoryTest extends SpringUnitTest {

    @Autowired
    private StudentRepository studentRepository;

    @Before
    public void setup() {
        importJSON("student", "src/test/resources/student.json");
    }

    @Test
    public void findStudentByName_should_return_student() {
        assertEquals("Thijs", studentRepository.findByName("Thijs").getName());
    }
}

Or you could do an Integration Test with Embedded MongoDB
public class StudentServiceIT extends SpringIntegrationTest {

    @Autowired
    private StudentService studentService;

    @Autowired
    private StudentRepository studentRepository;

    @Test
    public void create_should_create_new_student() {
        studentService.create("James Doe");
        List studs = studentRepository.findAll();
        assertTrue(studs.size() == 1);
        assertEquals("James Doe", studs.get(0).getName());
        assertNotNull(studs.get(0).getEnrollmentDate());
    }
}

Conclusion


I like both Embedded MongoDB and Fongo. I prefer Fongo to support unit tests. It's easy to setup, fast to startup. But for running integration tests I would suggest using flapdoodle.embed.mongo as you're actually running on a real MongoDB instance giving you the closest to real-life scenario.

8 comments:

  1. Very nice article, saved me a lot of time.
    I have ended up using de.flapdoodle.embed.mongo

    Regards,
    Dan

    ReplyDelete
  2. If you're using MongoDB and you want to start/stop the database as part of a Maven build, you can use this plugin: https://github.com/joelittlejohn/embedmongo-maven-plugin. It's a wrapper for the flapdoodle API.

    ReplyDelete
  3. Is it that you missed this dependency, or I am missing something?

    org.springframework
    spring-test
    XXX

    ReplyDelete
  4. The effectiveness of IEEE Project Domains depends very much on the situation in which they are applied. In order to further improve IEEE Final Year Project Domains practices we need to explicitly describe and utilise our knowledge about software domains of software engineering Final Year Project Domains for CSE technologies. This paper suggests a modelling formalism for supporting systematic reuse of software engineering technologies during planning of software projects and improvement programmes in Project Centers in Chennai for CSE.

    Spring Framework has already made serious inroads as an integrated technology stack for building user-facing applications. Spring Framework Corporate TRaining the authors explore the idea of using Java in Big Data platforms.
    Specifically, Spring Framework provides various tasks are geared around preparing data for further analysis and visualization. Spring Training in Chennai

    ReplyDelete
  5. This is the most important topic on mongodb, we are waiting for more article please share on MongoDB online training

    ReplyDelete
  6. please share source code, it doesn't work.

    ReplyDelete