Continuous integration is a practice the grew out of the extreme programming community, and was written about by Martin Fowler and Kent Beck. Martin Fowler’s paper provides a very nice definition of the term

Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily – leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly. This article is a quick overview of Continuous Integration summarizing the technique and its current usage.

When being used effectivly, the practice of continious integration provides feedback on the ‘health’ of the software after every commit to the software repository allowing developers to identify and fix defects early. Since the time between the commit and the identification of a problem is so short, bugs are often simpiler as eassier to fix.

Laying the out the guidelines

There are many continuous integration servers out there and they each have their own pros and cons, but just to shrink the playing field a little bit I’m going to layout some basic criteria. So I am looking for the following in a continuous server:

• Free, I know that there are numerous COTS solutions, but free should fit into everyone’s budget. Also with as many good free solutions available with you really don’t need a COTS product.
• Should work using existing projects build files
• Should work with a popular SCM system, I’m looking for at least SVN, CVS
• Should have an adequate selection of notification systems

I know that these are pretty vague but these criteria do eliminate many solutions from the playing field, and in my opinion the solutions that are left are going to be the more mainstream and stable solutions.

So here we go, in no particular order:

CruiseControl

CruiseControl is a product that I’ve had a love/hate relationship with for many years, while it does its job and possesses all the fundamental qualities that you would look for in an it is rather cumbersome to configure, and not the most refined thing to look at. That being said it is a work horse and it does get the job done.

Installation

The installation of cruisecontrol is pretty straight forward and about on par for what you might expect for any opensource application. The quickstart from their website reads as follows:

Unzip the release to a directory, like ~/cc-sanbox/cruisecontrol-bin-2.7.3

Check that the scripts cruisecontrol-bin-2.7.3/cruisecontrol.sh and cruisecontrol-bin-2.7.3/apache-ant-1.7.0/bin/ant have execution permission.

Run ~/cc-sanbox/cruisecontrol-bin-2.7.3/cruisecontrol.sh from the newly unzipped directory

In all honesty that is a pretty accurate assessment of the level of effort it takes to install the software and start the application.

Configuration

All the configuration for CruiseControl is done in with XML files, and there is no good utility for helping you setup a project other than the configuration reference, a ton of elbow grease, and maybe a few cold ones. I think that if you take a look at the configuration reference, it is pretty daunting at first glance, and second glance … right up to about the 100th or so. It is just a lot of information to take in, and while it is very well documented and makes perfect sense once you get the hang of it, it will take a bit of time and experimentation until you’ve hit that balance of functionality that you were hoping to achieve from the system.

User Interface

There are actually two user interfaces that come with CruiseControl.

The standard interface is kinda of a web 1.0 older looking interface, although it is powered off of XSLT stylesheets, and serverl other new(er) technologies, it just doesn’t bring in the easy to use kind of appeal to the product.

	This is the main status page, it is a fairly simple overview of all the projects that the server is tracking, the status of the build and the last time a build was run. It provides a link into the project so you can investigate any failures or access any artifacts produced by that build.
	This screen gives you a detailed view of a particular project. On the left you can pick which build you want to look at so you can compare the current build to a past build. This view is customizable if you have the stomach for editing the stylesheets, and you can add support for the output of any other build tool that can produce xml output like findbugs, checkstyle, pmd etc..

The dashboard interface is a bit better, it uses AJAX and provides a more modern feel.

	Like above this view gives you an overall sense of how your projects are doing, this view with the color-coded boxes allows you to keep an eye on the status of many projects and mousing over any box will give you more information like project name and last build time.
	Much like the screen in the old user interface, this allows you to see information like which tests are failing, what the changeset was that triggered the build. And you can navigate to previous builds.

While I do consider the dashboard interface an improvement over the standard interface, it still lacks a polished quality. But that aside it does allow you to do everything and anything you could want to do with it, you just might have to go into the source and do some tweeking.

Overall

While CruiseControl probably does everything that you would like it to do, its a bit hard on the eyes, and I talked with many users of it and they have all expressed the same feelings of mild irritation when it comes to configuring the server. All that being said it is the original so it gets points for that.

LuntBuild

Installation

Installation of LuntBuild is pretty straight forward, it has the option of an installation GUI.

Download Luntbuild installer from Luntbuild Sourceforge site, or from Luntbuid Javaforge  site
Run command java -jar luntbuild-xxx-installer.jar
cd 
java -jar luntbuild-standalone.jar localhost 8888

Now if you don’t mind it using teh HSQL DB embedded database, then you’re done, if you want it to use another type of database then there are instructions to set that feature up here.

Configuration

All the LuntBuild configuration is done within the web interface, select the projects tab and click the (new project) icon.

You can then start customizing the fields to your hearts desire. However most all of the fields are just text input boxes and its up to you to select the correct thing.

User Interface

	The main status view shows you the status of each project’s last build, this view also provides links into the project configuration and into the details of the last build.
	The build status view, shows you all the basic information about the status of the build. Not to many bells and whistles here just a basic view.

Overall

LuntBuild seems that it will get the job done, but it isn’t going to win any awards doing it, it seems to be a product that is just kind of middle of the road ‘eh’

Continuum

Continuum is a lightweight brought to you by the team who developed Maven. It is has a fairly simple installation and a novice amount initial configuration.

Installation

The installation for continuum is pretty straight forward

• Download the standalone version from the Download page
• Extract the file
• Set a JAVA_HOME environment variable which use a jdk >= 1.5
• configure JNDI resources (such as SMTP server, database, etc..)

It is actually this last step that seems the most complicated, because it involves mucking around in the jetty XML files to define the resources.

Configuration

	Screen that allows you to configure a new Ant project
	Screen that allows you to edit a project

User Interface

For my tastes the Continuum interface is a little daunting looking, it is defiantly utilitarian. But I don’t know that it is particularly user friendly. I prefer interfaces that use constrained input fields that are harder for new users to screw up.

Overall

While Continuum is a full featured server, I’m not sure that it would be very well used, the harsh nature of the user interface doesn’t really help users want to use it. I believe that Continuum would likely be deployed and then quickly fall into disrepair.

Hudson

Installation

You will never find a product that is as easy to install as Hudson. Just download the file and run

java -jar hudson.war

You will be off and running, no messing around in configuration files or anything. Now if you really want to mess with the configuration files (and you really don’t have to, because everything can be done in the UI) they are available and you can hack away to your hearts content.

Configuration

	This is just an example of one of the many screen you find to help you configure parts of the system, everything is in small manageable pieces and almost all (if not all) have examples and context help to show you what goes where,
	This is a view of available plugins that you can select and just press install and hudson will go out and upgrade itself, you can’t ask for more than that.

User Interface

	This is the main screen, it is simple and elegant and it tells you everything you want to know about all the projects as well as what the server is working on building now, and what is in the build queue.
	This is a typical screen that displays the project summary, you can see that the it is giving you the trend for unit test failures in the form of a graph which I love. Most of the project detail screen are similar to this one, favoring pictures over text so you can get all the information you need at a quick glance.

Overall

I love this product, installation is a snap and configuration is extremely simple but the most powerful thing about hudson is its plugin API it has allowed many developers (including your’s truly) to contribute to the project creating an impressive wealth of functionality.

And I don’t think that I’ve found a single thing that the other projects can do that hudson doesn’t do as well if not better.

Conclusions

Of course it depends on what you are looking for, cruisecontrol is going to be a bit more work up front but once you get it going, it will serve you well as long as you don’t mind its old school user interface. Whereas hudson will be up and running within minutes, and it boasts a very modern very easy user interface.

I’m not sure that I would waste my time with either LuntBuild or Continuum, they are both solid projects but neither presents a very compelling reason to choose them over CrusieControl or Hudson.

Since I believe that user interface is a critical piece of a continuous integration server I’m gonna give this one to Hudson. Since basically a CI server is something who’s whole purpose for existence is to nag you when you’ve done something wrong, and then help you narrow down what you did and when you did it. If you don’t have a descent interface then you developers will slowly start to ignore its feedback.

I’ve now been using AccuRev for about 6 months now, and I it seems that I am having to adjust my process to suit some of the idiosyncrasies of the tools. Now I’m not going to say that its a bad tool, because as with everything in computer science I’m certain that there was a specific use case that was identified that AccuRev fills perfectly, but I don’t have that use-case.

My primary gripe is the inability to update my local workspace while I have a dirty workspace. Now while I understand the rational for this, it irritates me to no end. When my team is working toward a release we are all working very closely and when somebody else completes a feature that I would find useful I would like to be able to checkout their changes without having to commit my changes. The AccuRev team seems to think that taking a course view of solves many of the SCM problems, and indeed it probably does .. for the tool designers, but what about the tool users?

In any decent software project, the software will naturally fallout into different modules, and I find it extremely useful to sometimes update a piece of the tree without updating other pieces. AccuRev doesn’t allow this at all, it forces you to update everything or nothing. This is the fundamental cornerstone of my problem with this product.  With any tool I expect to have some level of choice in how I will use that tool, and I fully understand that I am accepting the responsibility to utilize that tool properly else I only have myself to blame when damage is caused. In AccuRev everything is very one-dimensional and doesn’t allow me to work the way I want to work, and really who is supposed to be in control here, the human or the tool? (hint if your tool is AccuRev then you are not in charge)

Then there is this business with server workspaces, in AccuRev there are two levels for checking in a file, there is a ‘keep’ which stores the change in your private workspace on the server and then there is a ‘promote’ which puts the revision in the public stream to be shared with everybody else. The main problem with the approach is mostly psychological, having these two levels actually lengthens development iterations because people can check into their private workspaces for the duration of the development effort (I’ve seen weeks and months) and then only promote and merge into the public stream when they are completely finished, its very waterfall and not really helping anybody because it lengthens the integration and doesn’t provide the rest of your team with the benefits of your work until its too late. Now I will admit that this is a process problem, but it is something to consider because if you are working with inexperienced developers this might go unnoticed and then everybody will be in for a surprise when a very large merge happens and nothing works anymore.

At the end of the day AccuRev is a inflexible tool that comes with a pretty hefty price tag. I would rather use a free tool like subversion any day. While I’m certain that most of my complaints are petty, when you use a tool everyday its the little things that start to eat away at you. While I’m sure some of my frustrations are my own fault, I’ve tried to get to the bottom of all of them and have found no research to make me think otherwise.