When to replace Unit Tests with Integration Test


Its been a while I was thinking about integration vs unit testing. Lots of googling, questions in stack overflow and looking in many books. In this post I would like share with you the information I have found and what decision I arrive at this.


I think if you put this question to any Java developer, you will get an anwser supporting the unit test instantaneously. They will tell you that if you can not do unit testing, the unit of code you hava written is large and you have to bring it down to multiple testable unit. I have found these questions in Stack Overflow 1, 2, 3 and 4 useful in getting a good understanding of both terminologies.

Unit testing is widly followed in Java domain with most of the projects now also enforcing the code coverage and unit testing. Integration testing is relativly new and misunderstood concept. Even through it is practiced much less than, unit testing there can be varios uses of it. Lets take a simple user story and revisit both.

Lets consider it with a user story

User will give an INPUT in a UI form and when the submit button is clicked, the PROCESSED output should be shown in the next page. Since the results should be editable, both INPUT and OUTPUT should be saved in the database.

Technical Design

Lets assume that our application will grow rapidly in the future and we will design it now keeping that in mind. 


As shown in the above image it is a standard 4-tier design with view, service, dao. It has a application layer which contains the logic of how to convert the input into output. To implement the story we wrote 5 methods(1 in service, 2 dao to save input and output, 1 contains the business and 1 method in view layer to prepare the input.)

Writing some Unit test cases

If we were to unit test the story, we need to write 5 tests. As the different layers are dependent, we might need to use mock objects for testing. But apart from one method in application layer that does the original process, is there any need to unit test the other part? For example the method,

 public void saveInput(Input input){  
           Session session = sessionFactory.getCurrentSession();  
           session.save(input);  
 }  


When you unit test this, you will typically use a mock object of sessionFactory and the code willl always work. Hence I don't see much point in wiring a unit test here. If you observe carefully, apart from the application layer, all the other methods will be similar to what we have discussed.


What can we acheive with integration test?


Read here as a heads up for the integration test. As we have seen most of the unit tests for this story were not effective. But we can't skip testing as we want to find out the code coverage and make our code self testing. According to Martin flower in his article about Continuous Integration you should write the code that can test the other code. I fell the good integration tests can do this for you.


Lets write a simple itegration test for this situation,

 @Configurable(autowire = Autowire.BY_NAME)   
 @RunWith(SpringJUnit4ClassRunner.class)   
 @ContextConfiguration(locations = {"classpath:applicationContext.xml"})  
 public class SampleIntTest {  
      @Autowired  
      private SamService samService;  
      @Test  
      public void testAppProcessing(){  
      Input input = new Input();  
      //prepare input for the testing.  
      Output output = samService.processInputs(input);  
      //validate outpt gainst the given input.  
      if(!isValidate(input,output)){  
           Assert.fail("Validation failed!");  
      }  
 }  



I have skipped the view layer here as it not so relevent. We are expecting the INPUT from the UI in a bean Input and thats that we willl have in the service layer. Here, With this few line of coding, you can acheive the full functionality of the application from the service layer.


It is prefered to use a in-memory database like H2 for integration tests. If the table structure is complicated it may not be possible in that case,you can use a test instance of DB and prepare a script to delete all the data and run it as part of the test so that the DB state is restored back. This is important because in the next run the same data will be saved again.


Another advantage of integration tets is that if you are changing the implementation the test need not be changed because it ic concerned with the input and output.This is useful in refactoring the code without change in the test code. Also, you can schedule these tets to measure the stability of the appication and any regression issues can be caught early.


As we have seen, integration test are easy to write and are useful, but we need to make sure it does not replace the unit testing completely. When ever there is a rule base system(like the LogicalProcesser in our case) it should be mandatory because you can't cover all the scenarios with the integration test.


So as always JEE is about making choice asnd sticking to it. For the last few months we were practicing it in our teams and it is really going well. Currently we made integration test mandatory and unit tests and optional. Always review the code coverage and make sure you get good coverage in the core of the system(the LogicalProcesser in our case).

7 comments:

  1. This comment has been removed by the author.

    ReplyDelete
  2. (Oops, did a mistake)...

    Good article.

    I could add another way to realize the test with a classic (non memory) database. What we want is to keep the data unmodified after a test, and there an easy way to do it: do the test in a transaction that is always rollback.

    How to do it? Use the spring @TransactionalConfiguration annotation in your integration class. In this case, each integration test is done in a transaction that is rollbacked when the test is done, so the next test won't see what was modified.

    See : http://static.springsource.org/spring/docs/3.0.x/spring-framework-reference/html/testing.html#integration-testing-annotations

    I wanting to add that a memory database is still a better choice for performance reasons.

    ReplyDelete
  3. @funkygono : Thanks you for the information.

    ReplyDelete
  4. First of all, this is really a huge topic, that can not be addressed in one blog post (or a comment!). However, I would like to add just few words.

    I understand perfectly why you do not see much sense in writing of unit tests for saveInput() method and similar simple ones. I agree that writing tests AFTER the code is written (and probably tested manually) does not make much sense. All you get is kind of a copy of production code in your tests, which makes them break with every change you do to production code.

    Having said that, I have only two questions/comments, both boiling down to the same fact, that some things are much easier to be thoroughly tested with unit, than with integration tests:
    - what about exceptional situations? They are much easier to be simulated with unit tests than with integration tests. At some point you want to see how your application behaves when DB goes down, or web service responds with a rubbish, or etc. etc. It is a piece of cake with unit tests.
    - what about growing logic? I mean, one day saveInput will grow, it will do some parameters checking or do some other "business" activity. Then unit tests might be more then welcomed.

    In general it is hard to test all paths with integration tests, while it is more likely to be possible with unit tests. As stated previously this is a topic for a book rather for a blog post, so I do not think we will find a definite answer to the question we discuss.

    BTW. I omit completely the TDD approach, which is also something to consider here.

    --
    Regards,
    Tomek Kaczanowski
    Practical Unit Testing With TestNG And Mockito

    ReplyDelete
  5. Hi Tomek,

    Thank you for adding your thought on my article. One your two question I must agree it will be difficult to test an exceptional scenario or to handling the changing logic. But the point is in most the time we might not need both.

    In the mean while looking forward to your book in this topic!

    ReplyDelete
  6. Thank you so much for your post. This post really help me a lot and I have learnt some new things from your blog. I have bookmark your blog for future visit and here i am also sharing an article on integration testing hope you will like it.

    ReplyDelete
  7. software testing company in India
    Thank you for sharing such a nice post.
    very informative.
    please keep sharing.

    ReplyDelete