Copying Files Using NIO

Posted by pfranza
Share/Bookmark   October 25, 2008

Prior to the JDK 1.4 introduction of the NIO package a tipical file copy routine would look something like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

public static void copyFile(File in, File out) throws Exception {
    FileInputStream fis  = new FileInputStream(in);
    FileOutputStream fos = new FileOutputStream(out);
    try {
        byte[] buf = new byte[1024];
        int i = 0;
        while ((i = fis.read(buf)) != -1) {
            fos.write(buf, 0, i);
        }
    } catch (Exception e) {
        throw e;
    } finally {
        if (fis != null) fis.close();
        if (fos != null) fos.close();
    }
  }

With the introduction of the NIO package’s conecpt of channels we can rewrite the fileCopy routine as:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

public static void copyFile(File in, File out) 
        throws IOException  {
        FileChannel inChannel = new
            FileInputStream(in).getChannel();
        FileChannel outChannel = new
            FileOutputStream(out).getChannel();
        try {
            inChannel.transferTo(0, inChannel.size(),
                    outChannel);
        } catch (IOException e) {
            throw e;
        } finally {
            if (inChannel != null) inChannel.close();
            if (outChannel != null) outChannel.close();
        }
    }

However on windows platforms, you may have to replace

1
2
3
4

try {
            inChannel.transferTo(0, inChannel.size(),
                    outChannel);
        }

with

1
2
3
4
5
6
7
8
9

try {
           // magic number for Windows, 64Mb - 32Kb)
           int maxCount = (64 * 1024 * 1024) - (32 * 1024);
           long size = inChannel.size();
           long position = 0;
           while (position < size) {
              position += 
                inChannel.transferTo(position, maxCount, outChannel);
        }

when attempting to copy a file in excess of 64Mb.



Uncategorized, coding
No Comments

Simplify Unit Testing With Dependency Injection

Posted by pfranza
Share/Bookmark   October 6, 2008

After you read a bit about dependency injection you might say “So what? There isn’t much of a practical gain” and an the surface I actually agree with you, I mean how many instances are there where we really need to be able to create infinate class permutation without using subclassing? I believe the place where you are going to see dependency injection score major points is with your unit testing. The main reason is that you can provide mock implementations of all the dependencies and eliminate any vairance in their behaviour from the class you are testing.

For example, continuing from my previous post: with traditional composition we could test the Car class like this:

1
2
3
4
5
6
7
8

public class CarTest extends TestCase {
 
 public void testCar() throws Exception {
 Car c = new Car();
 assertEquals(.....);
 }
 
}

but the behaviour of the instance of car is really coupled to the behaviour of the AutomaticTransmission and GasEngine classes. If something breaks in the AutomaticTransmission class, more than likely our AutomaticTransmissionTest and CarTest cases will fail. This is slightly misleading because there isn’t really a problem with the Car class, and yet its test is failing because of the coupled behaviour to the AutomaticTransmission class. Using the dependency injection version of the class allows us to do something very different.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

public class CarTest extends TestCase {
 
 private static class MockTransmission implements Transmission {
   ...
 }
 
 private static class MockEngine implements Engine {
   ...
 }
 
 public void testCar() throws Exception {
 Car c = new Car(new MockTransmission(), new MockEngine());
 assertEquals(.....);
 }
 
}

Now the Car we are testing has no dependencies outside of our testcase, this allows us to tightly control what we are putting under test and allow us to focus our effort on testing the single class rather than any extra behaviour that comes from the subcomponenets. In this scenerio if something breaks in the AutomaticTransmission class only the AutomaticTransmissionTest will fail because CarTest no longer has a dependency on AutomaticTransmission allowing them to operate (and fail) independently of one another.



coding
No Comments

Dependency Injection Basics

Posted by pfranza
Share/Bookmark   October 6, 2008

Dependency Injection is a process of supplying external dependencies to componenets, changing the flow of control of the system to be inverted in comparison to the traditional architecture of software libraries. Dependency Injection builds of the concept that you should favor composition over inheritance [Effective Java Programming Language Guide, chapter 4, item 14], but extending it to say that the specific implementations of the subcomponenets that you are using to compose your class should be fully decoupled from the class itself.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

public interface BrakeSystem {
 
}
 
public interface Engine {
 
}
 
public class Car {
 
 private Transmission transmission = new AutomaticTransmission();
 private Engine brakes = new GasEngine();
 
}

We can see that in class Car we’ve coupled the implementation of AutomaticTransmission and DiscBrakeSystem to the implementation of Car. What do we do when we want to create an instance of Car that has a manual Transmission? The answer is dependency injection. In this example we find it trivial to remove the coupling of subcomponent implementations by passing in the subcomponents at time of construction rather than at the time of compiliation.

1
2
3
4
5
6
7
8
9
10
11

public class Car {
 
 private Transmission transmission;
 private Engine engine;
 
 public Car(Transmission transmission, Engine engine) {
    this.transmission = transmission;
    this.engine = engine;
 }
 
}

Now we can use the constructor to create variations of the cars behaviour.

1
2
3
4
5
6
7
8
9
10
11
12
13

public class CarFactory {
 
 public static final Car createAutomaticCar() {
 return new Car(new AutomaticTransmission(),
 new GasEngine());
 }
 
 public static final Car createManualCar() {
 return new Car(new ManualTransmission(),
 new GasEngine());
 }
 
}

Using this method we can vary the components of car without subclassing it, allowing the freedom to provide numerious unique representations of cars without changing anything in the car class itself. Its all left to wiring the classes together in an appropriate way.

Can’t I just use the Service Locator Pattern?

Well it is true that dependency injection does aim to solve the same problem as the Service Locator Pattern, usage of the pattern requires you to build in logic that preforms the dependency resolution and construction. This introduces cross cutting behaviour that disrupts the cohesion and singular responsibility of the class. By removing the service location logic and relegating it into a factory or some other framework level function, a high level of cohesion can be maintained. This will be true for both the object and its factory, as the single responsibility of the factory is to construct instances of the class, and the class just has to be itself.

Usage of Dependency Injection Frameworks

I sit firmly on the fence about the usage of frameworks for dependency injection, becuase it seems that they all have a dark side to them that in one way or another reduces readability/traceability of the classes. Some like Google Guice use method annotations to define where the injection points are, and while this seems to be be a relativly consice way to inform the framework it adds a lot of extranious information to your code that doesn’t really make sense within the context of the class.

This is where I actually like the use of factories that construct the objects and do the injection for you. Its generally clean, and it localizes the couplings to a factory which is natually where they should be. The downside is sometimes dependency injection can get several layers deep (you have a class that requires classes which requre classes etc) and the notion that you would have a factory method for each one of the permutations is quite excessive. However if you only add a method that builds the specific combination that you are looking for this might strike a nice middle ground.



Uncategorized, coding
No Comments

Comparison of Continuous Integration Servers

Posted by pfranza
Share/Bookmark   September 26, 2008

A Short Background

Continuous integration is a practice the grew out of the extreme programming community, and was written about by Martin Fowler and Kent Beck. Martin Fowler’s paper provides a very nice definition of the term

Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily – leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly. This article is a quick overview of Continuous Integration summarizing the technique and its current usage.

When being used effectivly, the practice of continious integration provides feedback on the ‘health’ of the software after every commit to the software repository allowing developers to identify and fix defects early. Since the time between the commit and the identification of a problem is so short, bugs are often simpiler as eassier to fix.

Laying the out the guidelines

There are many continuous integration servers out there and they each have their own pros and cons, but just to shrink the playing field a little bit I’m going to layout some basic criteria. So I am looking for the following in a continuous server:

  • Free, I know that there are numerous COTS solutions, but free should fit into everyone’s budget. Also with as many good free solutions available with you really don’t need a COTS product.
  • Should work using existing projects build files
  • Should work with a popular SCM system, I’m looking for at least SVN, CVS
  • Should have an adequate selection of notification systems

I know that these are pretty vague but these criteria do eliminate many solutions from the playing field, and in my opinion the solutions that are left are going to be the more mainstream and stable solutions.

So here we go, in no particular order:

CruiseControl

CruiseControl is a product that I’ve had a love/hate relationship with for many years, while it does its job and possesses all the fundamental qualities that you would look for in an it is rather cumbersome to configure, and not the most refined thing to look at. That being said it is a work horse and it does get the job done.

Installation

The installation of cruisecontrol is pretty straight forward and about on par for what you might expect for any opensource application. The quickstart from their website reads as follows:

Unzip the release to a directory, like ~/cc-sanbox/cruisecontrol-bin-2.7.3

Check that the scripts cruisecontrol-bin-2.7.3/cruisecontrol.sh and cruisecontrol-bin-2.7.3/apache-ant-1.7.0/bin/ant have execution permission.

Run ~/cc-sanbox/cruisecontrol-bin-2.7.3/cruisecontrol.sh from the newly unzipped directory

In all honesty that is a pretty accurate assessment of the level of effort it takes to install the software and start the application.

Configuration

All the configuration for CruiseControl is done in with XML files, and there is no good utility for helping you setup a project other than the configuration reference, a ton of elbow grease, and maybe a few cold ones. I think that if you take a look at the configuration reference, it is pretty daunting at first glance, and second glance … right up to about the 100th or so. It is just a lot of information to take in, and while it is very well documented and makes perfect sense once you get the hang of it, it will take a bit of time and experimentation until you’ve hit that balance of functionality that you were hoping to achieve from the system.

User Interface

There are actually two user interfaces that come with CruiseControl.

The standard interface is kinda of a web 1.0 older looking interface, although it is powered off of XSLT stylesheets, and serverl other new(er) technologies, it just doesn’t bring in the easy to use kind of appeal to the product.

This is the main status page, it is a fairly simple overview of all the projects that the server is tracking, the status of the build and the last time a build was run.

It provides a link into the project so you can investigate any failures or access any artifacts produced by that build.
This screen gives you a detailed view of a particular project. On the left you can pick which build you want to look at so you can compare the current build to a past build.

This view is customizable if you have the stomach for editing the stylesheets, and you can add support for the output of any other build tool that can produce xml output like findbugs, checkstyle, pmd etc..

The dashboard interface is a bit better, it uses AJAX and provides a more modern feel.

Like above this view gives you an overall sense of how your projects are doing, this view with the color-coded boxes allows you to keep an eye on the status of many projects and mousing over any box will give you more information like project name and last build time.
Much like the screen in the old user interface, this allows you to see information like which tests are failing, what the changeset was that triggered the build. And you can navigate to previous builds.

While I do consider the dashboard interface an improvement over the standard interface, it still lacks a polished quality. But that aside it does allow you to do everything and anything you could want to do with it, you just might have to go into the source and do some tweeking.

Overall

While CruiseControl probably does everything that you would like it to do, its a bit hard on the eyes, and I talked with many users of it and they have all expressed the same feelings of mild irritation when it comes to configuring the server. All that being said it is the original so it gets points for that.

LuntBuild

Installation

Installation of LuntBuild is pretty straight forward, it has the option of an installation GUI.

Download Luntbuild installer from Luntbuild Sourceforge site, or from Luntbuid Javaforge  site
Run command java -jar luntbuild-xxx-installer.jar
cd 
java -jar luntbuild-standalone.jar localhost 8888

Now if you don’t mind it using teh HSQL DB embedded database, then you’re done, if you want it to use another type of database then there are instructions to set that feature up here.

Configuration

All the LuntBuild configuration is done within the web interface, select the projects tab and click the (new project) icon.

You can then start customizing the fields to your hearts desire. However most all of the fields are just text input boxes and its up to you to select the correct thing.

User Interface

The main status view shows you the status of each project’s last build, this view also provides links into the project configuration and into the details of the last build.
The build status view, shows you all the basic information about the status of the build. Not to many bells and whistles here just a basic view.

Overall

LuntBuild seems that it will get the job done, but it isn’t going to win any awards doing it, it seems to be a product that is just kind of middle of the road ‘eh’

Continuum

Continuum is a lightweight brought to you by the team who developed Maven. It is has a fairly simple installation and a novice amount initial configuration.

Installation

The installation for continuum is pretty straight forward

  • Download the standalone version from the Download page
  • Extract the file
  • Set a JAVA_HOME environment variable which use a jdk >= 1.5
  • configure JNDI resources (such as SMTP server, database, etc..)

It is actually this last step that seems the most complicated, because it involves mucking around in the jetty XML files to define the resources.

Configuration

Screen that allows you to configure a new Ant project
Screen that allows you to edit a project

User Interface

For my tastes the Continuum interface is a little daunting looking, it is defiantly utilitarian. But I don’t know that it is particularly user friendly. I prefer interfaces that use constrained input fields that are harder for new users to screw up.

Overall

While Continuum is a full featured server, I’m not sure that it would be very well used, the harsh nature of the user interface doesn’t really help users want to use it. I believe that Continuum would likely be deployed and then quickly fall into disrepair.

Hudson

Installation

You will never find a product that is as easy to install as Hudson. Just download the file and run

java -jar hudson.war

You will be off and running, no messing around in configuration files or anything. Now if you really want to mess with the configuration files (and you really don’t have to, because everything can be done in the UI) they are available and you can hack away to your hearts content.

Configuration

This is just an example of one of the many screen you find to help you configure parts of the system, everything is in small manageable pieces and almost all (if not all) have examples and context help to show you what goes where,
This is a view of available plugins that you can select and just press install and hudson will go out and upgrade itself, you can’t ask for more than that.

User Interface

This is the main screen, it is simple and elegant and it tells you everything you want to know about all the projects as well as what the server is working on building now, and what is in the build queue.
This is a typical screen that displays the project summary, you can see that the it is giving you the trend for unit test failures in the form of a graph which I love. Most of the project detail screen are similar to this one, favoring pictures over text so you can get all the information you need at a quick glance.

Overall

I love this product, installation is a snap and configuration is extremely simple but the most powerful thing about hudson is its plugin API it has allowed many developers (including your’s truly) to contribute to the project creating an impressive wealth of functionality.

And I don’t think that I’ve found a single thing that the other projects can do that hudson doesn’t do as well if not better.

Conclusions

Of course it depends on what you are looking for, cruisecontrol is going to be a bit more work up front but once you get it going, it will serve you well as long as you don’t mind its old school user interface. Whereas hudson will be up and running within minutes, and it boasts a very modern very easy user interface.

I’m not sure that I would waste my time with either LuntBuild or Continuum, they are both solid projects but neither presents a very compelling reason to choose them over CrusieControl or Hudson.

Since I believe that user interface is a critical piece of a continuous integration server I’m gonna give this one to Hudson. Since basically a CI server is something who’s whole purpose for existence is to nag you when you’ve done something wrong, and then help you narrow down what you did and when you did it. If you don’t have a descent interface then you developers will slowly start to ignore its feedback.



coding, reviews
10 Comments

Little Endian Input Stream

Posted by pfranza
Share/Bookmark   September 26, 2008

As everybody probably knows, Java only supports Big Endian streams but every once and a while you find yourself in the predicament of wanting to interface with a legacy system that wants to use little endian encoding. So I’m attaching a class that will allow you to use the nice DataInputStream API but that will read the data in little endian format.

*On a side note if you don’t know the difference between big and little endian or what the etymology of the term is. Endianess is basically to do start reading from the left most or right most bit (assuming bits were arranged in this fashion) you can find more information here. The term comes from Jonathan Swift’s satirical novel Gulliver’s Travels where there was a large disagreement about which side of one’s soft-boiled egg you should open.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132

public class LittleEndianDataInputStream extends InputStream implements DataInput {
 
	public LittleEndianDataInputStream(InputStream in) {
		this.in = in;
		this.d = new DataInputStream(in);
		w = new byte[8];
	}
 
	public int available() throws IOException {
		return d.available();
	}
 
 
	public final short readShort() throws IOException
	{
		d.readFully(w, 0, 2);
		return (short)(
				(w[1]&0xff) << 8 |
				(w[0]&0xff));
	}
 
	/**
	 * Note, returns int even though it reads a short.
	 */
	 public final int readUnsignedShort() throws IOException
	 {
		 d.readFully(w, 0, 2);
		 return (
				 (w[1]&0xff) << 8 |
				 (w[0]&0xff));
	 }
 
	 /**
	  * like DataInputStream.readChar except little endian.
	  */
	 public final char readChar() throws IOException
	 {
		 d.readFully(w, 0, 2);
		 return (char) (
				 (w[1]&0xff) << 8 |
				 (w[0]&0xff));
	 }
 
	 /**
	  * like DataInputStream.readInt except little endian.
	  */
	 public final int readInt() throws IOException
	 {
		 d.readFully(w, 0, 4);
		 return
		 (w[3])      << 24 |
		 (w[2]&0xff) << 16 |
		 (w[1]&0xff) <<  8 |
		 (w[0]&0xff);
	 }
 
	 /**
	  * like DataInputStream.readLong except little endian.
	  */
	 public final long readLong() throws IOException
	 {
		 d.readFully(w, 0, 8);
		 return
		 (long)(w[7])      << 56 | 
		 (long)(w[6]&0xff) << 48 |
		 (long)(w[5]&0xff) << 40 |
		 (long)(w[4]&0xff) << 32 |
		 (long)(w[3]&0xff) << 24 |
		 (long)(w[2]&0xff) << 16 |
		 (long)(w[1]&0xff) <<  8 |
		 (long)(w[0]&0xff);
	 }
 
	 public final float readFloat() throws IOException {
		 return Float.intBitsToFloat(readInt());
	 }
 
	 public final double readDouble() throws IOException {
		 return Double.longBitsToDouble(readLong());
	 }
 
	 public final int read(byte b[], int off, int len) throws IOException {
		 return in.read(b, off, len);
	 }
 
	 public final void readFully(byte b[]) throws IOException {
		 d.readFully(b, 0, b.length);
	 }
 
	 public final void readFully(byte b[], int off, int len) throws IOException {
		 d.readFully(b, off, len);
	 }
 
	 public final int skipBytes(int n) throws IOException {
		 return d.skipBytes(n);
	 }
 
	 public final boolean readBoolean() throws IOException {
		 return d.readBoolean();
	 }
 
	 public final byte readByte() throws IOException {
		 return d.readByte();
	 }
 
	 public int read() throws IOException {
		 return in.read();
	 }
 
	 public final int readUnsignedByte() throws IOException {
		 return d.readUnsignedByte();
	 }
 
	 @Deprecated
	 public final String readLine() throws IOException {
		 return d.readLine();
	 }
 
	 public final String readUTF() throws IOException {
		 return d.readUTF();
	 }
 
	 public final void close() throws IOException {
		 d.close();
	 }
 
	 private DataInputStream d; // to get at high level readFully methods of
	 // DataInputStream
	 private InputStream in; // to get at the low-level read methods of
	 // InputStream
	 private byte w[]; // work array for buffering input
}

Download: LittleEndianDataInputStream.java

enjoy



Uncategorized, coding
5 Comments
← Previous Page
Next Page →