Continous testing with AutoTest.Net

It's about time for me to do some writing about AutoTest.Net. This is a project I have been working on for the last 2-3 months. It's a continous testing tool for .Net originally based on ruby's autotest. After playing with Ruby for a couple of evenings I really enjoyed the way you could work with it. My usual work circle of "write code, build, wait, run tests, wait" was replaced with "write code, save". Luckily the code I write tends to work more often than it breaks so waiting for builds and tests to run is a waste of time. Specially when I build and run tests about every 30 seconds or so.

So after the joy of working like that in Ruby I was determined to find a tool like that for .Net. After a bit of searching I found AutoTest.Net. The project was hosted at code.google.com and was initiated by James Avery but because of not having enough time on his hands the project was put on hold. I really wanted a tool like that so I went and got his permission to continue the project. It's now hosted on github.com/acken/AutoTest.Net. Today it contains support for both .Net and mono and it's cross platform. NUnit, MSTest and XUnit are the testing frameworks supported today and MbUnit will be added soon. It supports running tests from multiple testing frameworks in the same assembly.

Now, how does it all work you say? The whole thing consists of a console application and a winforms application. By now I use the winforms app about 98% of the time so that's what I'm going to show here.

The first thing you do ofcourse is to go to this link and download the latest binaries and unzip them to the folder of your choice. Locate the file named AutoTest.config and open it in your favorite xml editor. Now let's edit a few settings:

1. DirectoryToWatch: Set this property to the folder containing the source code you want to work with. AutoTest.Net will detect changes to this folder and it's subfolders.
2. BuildExecutable: This is the path to msbuild or in mono's case xbuild. You have the possibility to specify a version of msbuild pr. framework or visual studio version.. For now let's just specify the default <BuildExecutable> property. Something like C:\Windows\Microsoft.NET\Framework\v3.5\MSBuild.exe.
3. Now let's specify a testing framework. You can pick from zero to all though zero wouldn't be any fun. I'll go with NUnit in this example.
4. The last thing we want to do is to specify a code editor (<CodeEditor>). Let's pick visual studio. We can pass visual studio the file to open and the line to go to. Sadly there's a bug in visual studio preventing it to go to the right line :( So for now we'll rely on Ctrl+G. Anyways the config has visual studio set up correctly by default. Just make sure the path to devenv.exe is the same as on your machine.

Now we're ready to start the AutoTest.WinForms.exe application and do some real work.The first thing you 'll see is a form looking like this.

The only interesting thing right after startup is the button in the top right corner. As you can see now (gonna do something about the colors) the button is yellow. Behind this button you'll find the status for AutoTest.Net application. It's yellow now because the configuration has generated a warning. If the button is red an error has occurred within AutoTest.Net. Right now the window will look like this.

So let's go ahead write some jibberish in one of the files inside the folder we're watching and save the file. AT.Net should start working right after you save the file.

And of course jibberish means errors which will result in this.

When selecting one of the lines in the list you'll get the build error/test details underneath and you can click on the links to open the file in visual studio. Now let's fix the error we just made and save the file and let's see what happens.

And as expected it goes green with 5 succeeded builds and 221 passed tests. That's basically it. From here on it's lather, rinse, repeat.

Right now it's in alpha and it will of course have some bugs here and there. I hope this post will tempt you to try it out. Even at an early stage like this it's a really effective way of working!