Taking a broader view of what test-driven development is
As test-driven development (TDD) becomes more popular, it was inevitable that some people would become dogmatic about what is and what isn’t TDD, and assert that if you don’t do TDD exactly their way, you’re not doing TDD at all.
Even some critics of TDD take a narrow view of what TDD is. But if you examine what they do instead of TDD, you might find that they actually do is not the most dogmatic form of TDD, but it’s still TDD.
No one has appointed me the arbiter of what TDD is, and as far as I know, no one else has been appointed either. If anyone can claim that authority, that would be Kent Beck. Except that he says he did not invent TDD, but merely rediscovered it.
The way I see it, TDD is any use of tests in an automated testing framework, like JUnit for Java, to guide the process of programming. A robust, fully working program gradually emerges through the TDD process.
That might seem overly broad, but it clearly excludes testing after the fact. Writing unit tests for a program you already believe to be complete, merely to verify that it works correctly, that’s not TDD.
This definition also excludes experiment-driven development, in which one tries out different things in a Read Evaluate Print Loop (REPL) and incorporates what works into a program. However, a REPL can be a great way to help you think about tests you might want to write.
Also excluded from TDD is testing a few representative cases in a REPL or a “console program” as you work out the algorithm, then using automated testing to test hundreds or thousands of cases.
A big difference between TDD and testing after the fact is the meaningfully failing first test in the testing framework. It’s a test that fails not because of a compilation error, or a logical error in the test, or any reason other than that the correct algorithm is not present in the source yet.
Testing after the fact can feel like a chore. You will definitely make mistakes writing tests after the fact. Tests will fail even though the source is correct, and then it’s difficult to tell whether the problem is in the source or in the tests.
You will definitely also make mistakes in TDD, but the process leverages those mistakes to move you forward, and towards the correct solution.
To illustrate these concepts with a concrete example, I will use the example of fractions, like 1/2 and 22/7, which are easy enough to understand for most people. We will implement them in Java, using JUnit as our testing framework.
If we’re going to go to the trouble of defining a Fraction class, the main thing we want it to do is perform arithmetic on fractions symbolically rather than numerically.
So, for example, 1/3 + 1/7 = 10/21 precisely, not 0.47619047619048 nor whatever other arbitrarily truncated or rounded value floating point precision might give us.
My first instinct would be to write the bare minimum necessary for a syntactically valid constructor, and stubs for the arithmetic functions. Something like this:
public class Fraction { private final long numer, denom; // STUB TO FAIL THE FIRST TEST
public Fraction plus(Fraction addend) {
return new Fraction(0, 1);
} // STUB TO FAIL THE FIRST TEST
public Fraction minus(Fraction subtrahend) {
return new Fraction(0, 1);
} // STUB TO FAIL THE FIRST TEST
public Fraction times(Fraction multiplicand) {
return new Fraction(0, 1);
} // STUB TO FAIL THE FIRST TEST
public Fraction divides(Fraction divisor) {
return new Fraction(0, 1);
} public Fraction(long numerator, long denominator) {
this.numer = numerator;
this.denom = denominator;
}}
Then I would go ahead and write the tests. The most dogmatic adherents of taking Kent Beck literally might say that we must write a single test first, before anything else.
@Test
public void testAdd() {
Fraction addendA = new Fraction(1, 3);
Fraction addendB = new Fraction(1, 7);
Fraction expected = new Fraction(10, 21);
Fraction actual = addendA.plus(addendB);
assertEquals(expected, actual);
}Writing in Notepad, this might be fine. But in an integrated development environment (IDE) like NetBeans or IntelliJ, we probably have at least four errors because the IDE doesn’t know what this Fraction business is about.
Before we can have our failing first test in the FractionTest test suite, we need to define the class Fraction with at least plus() in it.
Nor can we put off writing a constructor because the tacit constructor takes no parameters, but obviously we need a constructor that takes a numerator and a denominator for its parameters.
Of course the computer is not going to explode just because we take our time in clearing the errors and warnings in the IDE.
However, writing a test before writing a stub limits the IDE’s ability to be helpful to us. I don’t mind typing “Fraction” over and over again, but I certainly appreciate an IDE auto-completing longer class names for me (IntelliJ is more helpful than NetBeans in this department).
Nor can any IDE read my mind and auto-complete “addendA.plus(addendB)” if I don’t already have at least some placeholder for plus() in Fraction. Otherwise, I can type “addendA.” and then NetBeans suggests plus(), and it might even fill in addendB as the operand.
Also, by writing stubs in the source class, the IDE can automatically generate some pretty good stubs in the test class, enabling us to concentrate on how to test our emerging class and not worry too much about what dependencies we need to carry over from source to test.
This is not a big deal for a simple example example like Fraction, but it is much appreciated when the object-oriented design involves inheritance across several standard and third-party packages.
Aside from auto-complete and automatic draft test generation, at this point, it really hasn’t made a difference whether we wrote the test first or the stub first. We can’t have a meaningful test failure with just one or the other.
Some programmers worry about the IDE being a crutch. It’s nice if you can type your source in a plaintext editor, and compile on the command line, but your client’s probably not going to care about that.
We have a much more pressing problem in our Fraction example: since we haven’t defined Fraction.equals() yet, testAdd() will fail even if we correct plus(), because our program thinks all Fraction objects are different.
A little housekeeping note: Of course our equals() override is not a static function; I just need to distinguish it from Object.equals(). This also applies to the others inherited from Object.
The test failure message might as well be nonsense. Something like this:
expected:<fractions.Fraction@cac736f>
but was:<fractions.Fraction@5e265ba4>So we must override equals(), and then the IDE will strongly recommend we also override hashCode(). And we also need to override toString(), and write tests for all these.
Since Fraction inherits equals(), hashCode() and toString() from Object, we can write tests for those without first writing stubs in the source, and still enjoy the benefits of auto-complete in an IDE.
@Test
public void testEquals() {
Fraction numberA = new Fraction(5, 8);
Fraction numberB = new Fraction(10, 16);
assertEquals(numberA, numberB);
Fraction numberC = new Fraction(4, 7);
assertNotEquals(numberA, numberC);
SQLNonTransientException obj = new
SQLNonTransientException();
assertNotEquals(numberB, obj);
} // TODO: Flesh out testHashCode()
@Ignore
@Test
public void testHashCode() {
fail("Pending test");
} @Test
public void testToString() {
Fraction fract = new Fraction(56, 58);
String expected = "28/29";
String actual = fract.toString().replace(" ", "");
assertEquals(expected, actual);
fract = new Fraction(3, -2);
expected = "-3/2";
actual = fract.toString().replace(" ", "");
assertEquals(expected, actual);
}
If you’re following along in your favorite Java IDE, you will need to import SQLNonTransientException from the java.sql package (no need to import the whole package, though). You might also need to import org.junit.Ignore.
I picked SQLNonTransientException for the third assertion in testEquals() because it’s obviously something that should never be equal to an instance of Fraction.
Almost anything else in the standard Java packages will do for this purpose. Or maybe you could use something from JUnit.
These tests don’t require Fraction to internally hold the numerator and denominator in lowest terms from the moment of object construction, though that’s probably a good idea.
These tests do now require the Fraction constructor to have some sense of what numerators and denominators are.
Someone might object that testEquals() is not granular enough and therefore this is not proper TDD. They might even go so far as to say this is not TDD at all.
If we run the tests right now, testEquals() will fail on the equality assertion, thus preventing the test runner from evaluating either of the inequality assertions.
We could very easily break up testEquals() like this:
@Test
public void testEquals() {
Fraction numberA = new Fraction(5, 8);
Fraction numberB = new Fraction(10, 16);
assertEquals(numberA, numberB);
} @Test
public void testNotEqualsOtherFraction() {
Fraction numberA = new Fraction(5, 8);
Fraction numberC = new Fraction(4, 7);
assertNotEquals(numberA, numberC);
}
@Test
public void testNotEqualsOtherObject() {
Fraction numberB = new Fraction(10, 16);
SQLNonTransientException obj = new
SQLNonTransientException();
assertNotEquals(numberB, obj);
}
If we run these tests right now, this more granular testEquals() will fail, but testNotEqualsOtherFraction() and testNotEqualsOtherObject() will both pass, depriving us of the meaningful initial test failure for those two.
With the “chunky” testEquals(), we could run the tests, see testEquals() fail, and then add an Object.equals() override to Fraction such that it is always true.
@Override
public boolean equals(Object obj) {
return true;
}For now we should ignore the IDE’s suggestion to generate the missing hashCode().
Run the tests. Now testAdd() should pass, though for the wrong reason (since Fraction.equals() is always true). And testEquals() should fail on the second assertion, since 5/8 ≠ 4/7.
Values should be different. Actual: fractions.Fraction@3830f1c0To amend Fraction.equals() so that testEquals() can get past the second assertion, we need to make equals() actually look at what the numerators and denominators are.
@Override
public boolean equals(Object obj) {
Fraction other = (Fraction) obj;
return (this.numer == other.numer &&
this.denom == other.denom);
}If you want, you can go ahead and let the IDE generate Fraction.hashCode() now. I think maybe we can still hold off on overriding Object.toString()…
NetBeans helpfully tells us that Fraction.equals() is not checking what type obj is. But we’re trying to do test-driven development, not IDE-driven development. So run the tests.
Now testAdd() should fail for the right reason, and testEquals() should fail on the third assertion.
expected:<fractions.Fraction@2fa6> but was:<fractions.Fraction@2df8>Hmm… actually, I’m not sure testEquals() really did fail on the third assertion. I was expecting it to cause some kind of bad cast exception.
Maybe we really do need to override Object.toString(), even if we don’t yet worry about making testToString() pass. So write Fraction.toString(), it should be something that will give us informative messages in the Test Results window but not good enough to pass testToString() just yet.
@Override
public String toString() {
return Long.toString(this.numer) + " / " +
Long.toString(this.denom);
}Run the tests again. Now we see the real reason testEquals() failed:
expected:<5 / 8> but was:<10 / 16>I don’t really like the spacing, but I think the tests should not require any specific spacing (including no spacing at all) to pass.
So testEquals() did not fail on the second or third assertion, but the very first assertion, since Fraction.equals() failed to recognize that 5/8 = 10/16.
We could amend Fraction.equals() so that it makes sure both the this fraction and the obj fraction are in lowest terms. But like I said earlier, it would be more efficient to put the fractions in lowest terms in the constructor, and then there is no need to do it again in Fraction.equals().
So now go ahead and rewrite the constructor so that it computes the greatest common divisor (GCD) of numerator and denominator, and uses that divisor to express the fraction in lowest terms.
I’m not including a source listing for this step. We need to implement some kind of GCD algorithm, making sure to test if it’s not something we know to have already been tested. But that would be too much of a sidetrack from the Fraction example.
Don’t yet worry about making sure the denominator is positive. Run the tests. Almost all of them should fail (testToString() should fail on the assertion for −3/2), but the third assertion of testEquals() should now cause an error:
Testcase: testEquals(fractions.FractionTest): Caused an ERROR
java.sql.SQLNonTransientException cannot be cast to fractions.Fraction
java.lang.ClassCastException: java.sql.SQLNonTransientException cannot be cast to fractions.Fraction
at fractions.Fraction.equals(Fraction.java:37)
at fractions.FractionTest.testEquals(FractionTest.java:97)This is good. The first and second assertions are passing, and the third assertion is failing for the right reason.
Now go ahead and correct the deficiency in Fraction.equals(). It seems that although NetBeans can auto-generate equals() from scratch, it is not capable of filling in what it knows to be missing. So you’ll have to go ahead and fill in something like this:
@Override
public boolean equals(Object obj) {
if (!(obj instanceof Fraction)) {
return false;
}
Fraction other = (Fraction) obj;
return (this.numer == other.numer &&
this.denom == other.denom);
}Of course in IntelliJ it is almost always very easy to run a single test by itself. We could have run the more granular testEquals() by itself, seen it fail, and then made it pass with by having equals() just give true regardless.
Then we could have ran testNotEqualsOtherFraction() by itself, seen it fail, done the bare minimum to make it pass, and then gone on to do the same for testNotEqualsOtherObject().
I’m guessing that as we work to make testNotEqualsOtherFraction() pass, IntelliJ will offer to add the missing instanceof check. But here let’s also hold off on that until we have testNotEqualsOtherObject() failing for that reason.
One could say that IntelliJ encourages a purer form of TDD. But to an observer watching both scenarios described here, what would be the most obvious difference between the two approaches?
With the first testEquals() in NetBeans, we started out with several yellow triangles and one gray circle in our Test Results window, and then we worked to make one of the yellow triangles turn to a green circle with a checkmark.
With testEquals() broken up into three smaller tests in IntelliJ, we worked towards making each of those turn to a green circle with checkmark. And then if we chose to run the whole test suite, we would have had at least three green circles, so at least two more than doing it the other way.
In both scenarios we used the test results to guide what we would do next, and at one point in both scenarios we ignored a warning from the IDE.
In real life you might not always follow the theoretically best path from idea to program.
But with TDD, that won’t always matter, as TDD will usually guide you to a robust program. In the example here, it would have made more sense to have started with testEquals() first and foremost, before trying to do anything with testAdd().
If your program has to have a certain feature, then you write a test for it. And if it doesn’t need that feature, you don’t write a test for it.
When a test passes the first time, it could be because you got ahead of yourself earlier on in the process. And that’s okay, as long as almost all the other tests went from meaningful failures to meaningful passes.
Taking a dogmatic, narrow view of TDD is almost antithetical to TDD, which is more about going with the flow, and trusting the flow to put you on the right track.