Wednesday, August 29, 2018

"Unit Testing" and Third Party Software

It is legit not my intention to make this a "testing" blog, but as I started this, I found myself for the first time in a testing role, so this is stuff at the top of mind. One thing I want to talk about, though, is "What is 'Unit Testing'?"

So one of the prime directives from the "Unit Testing" world is "Don't test software that isn't yours". This is a fine idea but there is are trap around it into which you don't want to fall. One specifically I want to discuss here:

Your configuration information IS YOUR SOFTWARE.

Let's pick an easy example: Hibernate. If you are building Java software, some form of JPA, and probably Hibernate is in your stack.

So lets talk about queries. Maybe you are using something with a dynamic proxy system. Maybe you are using compiled queries directly with your EntityManager. Doesn't matter. Your ANNOTATIONS are code, and should be tested.

Do you have to test the dynamic proxy generation? No. But if you have a DAO that looks like:

@Query("SELECT o FROM Foo WHERE o.value like '%:bar%')

List<Foo> fooValuesWithBar(String bar)

Should you be writing unit tests around whether the dynamic proxy correctly interprets your query? No. "Noy my yob mah." But making sure that all the configuration information in the annotation you wrote is correct is your job. If you are not writing a unit test that covers the annotation as code, you don't really have coverage.

The long and the short of this, is if you are using a tool to dynamically generate DAOs and your unit tests aren't going all the way to the database, you aren't actually covering your code, because the metadata about how the DAO framework will construct your queries is your code. Again, let's consider the unit test you should have around fooValuesWithBar()...  If you insert a stray character into the @Query, then your unit test should fail. The fact that you aren't writing the actual implementation of fooValuesWithBar() on this interface doesn't matter. You have defined the functionality with the annotation, so you need a test around it. The annotation is code.

So let's not just bitch about it, let's solve some problems.

So if you are using Hibernate/TopLink/EclipseLink, your code should be database portable. Is it? Do you care if it is not? At my current gig we are using Flyway as part of Spring Boot to do database migrations, but that involves writing SQL files. As soon as you get into writing SQL files, you have given up on DB portability. That said, the other option is trusting the JPA provider to update your schema. As much as people SAY that is a thing that can happen, I personally don't trust it.

That said, for the purposes of unit tests, there is no reason you can't rely on your JPA provider to create a schema it thinks is reasonable. That is, you can write DAO/Entity tests against in-memory Hypersonic/Derby/JavaDB and feel those are good tests. Database migration with a tool like Flyway can still be a thing, but you can pass that off to an "integration test" without feeling like you have lost something...  mostly.

So let's go with some rules:


  1. Don't mock a DAO unless you REALLY know what you are doing. Mocking things that are loaded with configuration is, IMHO, fraught. Does your test provide value? Well, that assumes the things you are mocking comply with production systems. Mocking an external service for which you have a contract test is OK. Mocking a DAO/Service/Other Dynamic Proxy where you aren't sure your annotations are correct?
  2. Use your DAO to persist and read outside of your test, rather than use verify calls. Something in a database is real. save(any(Foo.class)) is a crutch. Create a transient database if you need to.
  3. This doesn't just apply to DAOs. Anything with Annotation-specified behavior should be unit tested. This means custom XML/JSON (de)serialization rules, too.

No comments:

Post a Comment