Testing through interfaces

When at Vagif Abilov's BBQ while talking about testing Greg Young said something like "I wonder what tests would look like if we wrote tests against interfaces instead of their implementation". And really, what would they look like?

The scenario was that you have an interface that has multiple implementations and you would want to write tests against the interface testing all implementations. This would seriously reduce the number of tests you would have to write. So let's give it a try.

The first thing we need to keep in mind is that we're writing the tests on something abstract. This means that we don't really know what to expect. When passing x in to the various implementations it's not certain that all of them will answer y. Actually hopefully only one of them would answer y where x is act and y is assert. If not there would be multiple implementations doing the exact same thing and that would kind of defeat the purpose. Off the top of my head that leaves us with the following scenarios:

X stays the same while Y varies
This would be something like an ICalculator. The ICalculator would have implementations like DecimalCalculator and OctalCalculator. When running tests here we would end up with results like this:

• DecimalCalculator: 7*7 = 49
• OctalCalculator: 7*7 = 61

Which means that when writing these types of tests we need to be able to handle asserting on certain values pr. implementation.

X varies while Y stays the same

Let's imagine that we have some type of parser taking xml in returning a list of objects of a certain type. This would typically mean one implementation pr. xml schema while the output could be the same. So writing these types of tests  we'll have varying code for passing parameters while the assert would stay the same.

Ok, that wasn't too bad. We could probably make this look clean. Now over to some other aspects that we'll have to deal with.

Dependencies

With the right (wrong) implementation faking dependencies might be a hellish thing with this solution. I guess that's a good thing as it forces us to not make a mess of it. But still we need some way of handling setting up dependencies for the implementations.

Resolving implementations

We need a way to retrieve all implementations for an interface. Of course this is something we do all the time with DI containers so any DI container would provide us with what we need here. We could probably do something smart here to inject the faked dependencies we'll need for each implementation.

With this in mind let's set up a test for the calculator scenario. The first thing I did was creating a class for handling the plumbing. Right now this class takes care of resolving all implementations of the chosen interface, running the test on each implementation and performing specified assertions. My test ended up looking like this:

        [Test]
        public void Should_multiply()
        {
            var tester = new InterfaceTester<ICalculator>();
            tester.Test(c => c.Multiply(7, 7))
                .AssertThat<DecimalCalculator>().Returned(49)
                .AssertThat<OctalCalculator>().Returned(61);
        } 

I'm quite happy with that. This test is both extend able and readable. Now let's do the same to the scenario with the string parser. I'll just extend the plumbing class used in the previous example to handle varying input parameters. The implementation ended up looking like this:


        [Test]
        public void Should_parse_number()
        {
            var tester = new InterfaceTester<INumberParser>();
            tester
                .Test<XmlParser>(x => x.Parse("<number>14</number>"))
                .Test<StringParser>(x => x.Parse("14"))
                .Returned(14);
        }


I can't say I'm as happy with this one as the complete delegate is copied for both implementations and not just the part that differs. But still it's a huge simplification compared to writing a full test suite pr implementation.

I guess I'll leave it at that for now. What this does not cover is setting up dependencies which likely will complicate the implementation a bit. After doing this implementation I can really see the value of writing my tests like this. It would save me time and energy and would leave me with a cleaner simpler test suite. The implementation ended up being fairly simple. Initial conclusion: Writing tests against interfaces is a good idea!

I'd love to hear  your thoughts on this! And if you're interested in the full source code let me know and I'll upload it to github or something.