My journey with unit testing in Java so far
I am a very recent to the unit testing bandwagon. Perhaps it seemed way too corporate for my taste when I first read about it. But now I’ve come to see that it can be useful even for someone working alone on a hobby Java project.
The first time I read about unit testing, I thought it was kind of pointless and stupid. You write a program that does one thing, and then you write another program that does the same thing the same damn way?
I realized early on that it’s possible to fool yourself with unit testing. I didn’t see the point back then.
Getting the green bar with the 100.00% on it, or a column of green checkmarks, that’s not the point of unit testing. Because if that’s all you want, you can just do a few tests like this one:
@Test
public void fakeTest() {
System.out.println("Fake test");
}Nor is the red bar with 0.00% the point either, because for that you can just run auto-generated tests without making any changes.
The point is, according to my understanding now, to put each individual component of the program through the wringer, so that any problems arising in the system as a whole are due to an unexpected interaction, and not a flaw in any of the individual components.
When I was starting to learn about unit testing, I was also starting to work in earnest on a Java program to draw certain mathematical diagrams pertaining to prime numbers in certain domains.
I’m not going to go too in depth on the math here. It’s not that it’s advanced, but it does take several paragraphs to explain properly and it would be a major sidetrack from the topic of unit testing. I will try to limit the math stuff to basic high school math.
A lot of it boils down to checking whether ordinary, “simple” whole numbers are prime or not, and then drawing diagrams to show the results of the calculations.
Two of the diagrams my program would draw are famous. Not Mandelbrot set famous, but famous enough that a Google image search can readily bring them up. Look up “Gaussian primes” or “Eisenstein primes.”
These correspond to the “discriminants” −1 and −3 in my program (the terminology is incorrect but convenient).
I can just compare the results of my program to the images Google shows me. As for −2, −5, −6, −7, etc., I can visually scrutinize the results of my program to make sure they make sense.
At that early stage, it occurred to me that the program could also draw diagrams of sets mathematicians call “ideals,” and that some of the functions could be used to explore the Euclidean GCD algorithm in domains that are “not Euclidean.”
For that latter application, it does not immediately occur to me what sort of diagram should be drawn. I would have to be sure that the basic arithmetic functions on “complex” numbers (addition, subtraction, multiplication and division) all work correctly.
The prospect of slowly inputting a bunch of numbers at the command line and carefully checking the results one by one was not the least bit appealing. If only there was a way to have the computer test those functions on several numbers in quick succession and let me know if the results are correct or not!
That’s how I realized the program I was working on could use unit testing after all.
If you’ve read this far, it might seem like a fair assumption that you know what unit testing is. Also, I will assume you know at least the basics of Java or C#.
Even so, an elementary explanation of what unit testing is might help avoid confusion with the related terminology. I will put a few key terms in bold.
Unit testing tests whether the individual components of a program work correctly in isolation. Unit testing is not about testing how well a program works with other programs or devices, and it’s certainly not about testing how intuitive the user interface is to a human user.
Unit testing is not unique to Java, which is part of the reason I’m going to use the term “subroutine” to mean what would be called a “method” in Java, or a function or procedure in the old Pascal programming language.
Plus, as an added bonus, the word “subroutine” has somewhat the flavor of Treknobabble. Though I think it still makes sense to use the term “function” to refer to methods with any return type other than void. Call me old-fashioned.
So you write a subroutine in C++, C#, Java, whatever, and then you write another subroutine in that same programming language which calls on the first subroutine and checks the validity of the results.
The second subroutine is a test subroutine, which we just call a test. A test is any subroutine that serves no other purpose than to test the operation of another subroutine and report whether or not the subroutine worked correctly.
Let’s say a subroutine calls on another subroutine, checks the validity of the subroutine’s output and does some System.out.println() to report on the performance of the first subroutine. That is indeed a test.
But a test is more useful when it contains one or more assertions, which makes it easier for the computer to tell whether the test passed or not. If all the assertions in a test hold true, the test passes, but if even just one assertion is false, the whole test fails.
Perhaps the most common assertion is that two things are equal. Here is a toy example of an equality assertion:
int expResult = 2;
int result = 1 + 1;
assertEquals(expResult, result);That’s a toy example because the assertion should always hold true. If it’s false it might signal a very peculiar problem with the hardware.
Here is another example of an equality assertion, one likelier to reveal a mistake the programmer can actually fix:
GaussianInteger expResult = new GaussianInteger(-1, 0);
GaussianInteger imagUnit = new GaussianInteger(0, 1);
GaussianInteger result = imagUnit.times(imagUnit);
assertEquals(expResult, result);The assertion should hold true if the times() function of GaussianInteger has been properly defined.
The program you’re writing probably consists of several subroutines, which are likely grouped in classes and packages, and maybe each subroutine needs its own test. More than one test to be run together makes a test suite.
If you have a class of tests with a test for each subroutine in another class, it makes sense to call the class of tests a test class, and to name it accordingly. For example, a test class for class Dashboard would quite logically be class DashboardTest.
A testing framework makes it easier to group tests together and run them in different combinations one after the other. The framework then reports the results of the tests, usually as a percentage, with a 100% pass rate generally associated with the color green, and anything less with the color red.
Perhaps the most famous testing framework for Java is JUnit, though TestNG is gaining ground. For C#, there is XUnit; a little thought will show why they did not use the letter C for that name.
It should be emphasized that the framework will most likely not run the tests in source code order; I will harp on this point a bit later on.
I hear TestNG has some facility for specifying test order. As far as I know, the most you can do in JUnit is designate code to execute before or after the test class or before or after each test.
One time you might have the framework run all the tests for one class. Another time you might have it run all the tests for one package, or all the tests for one project.
This enables you to break up programming and testing into manageable chunks. Let’s say one package in the project is to consist of five classes. You can write one class, then write the test suite for that class, and test and debug that one class before moving on to the other classes and their tests.
Or you can even write all the tests first and then write the classes to be tested. That’s test-driven development, which has its pros and cons, but I’m not going to say too much more about it today because I don’t program that way (though I suppose I might have to if I were to get a job in a test-driven shop).
You can write tests from scratch or you can have your integrated development environment (IDE), like Eclipse or NetBeans, automatically generate tests for you which then you review and tweak, or sometimes almost completely rewrite.
I can only guess as to the granularity of automatically generated tests on other IDE and testing framework combinations, but at least NetBeans with JUnit seems to generate one test per subroutine. For example, for boolean primeQ(int number) it might generate public static void testPrimeQ().
But nothing prevents you from writing more tests for different aspects of the same subroutine. For example, in addition to testPrimeQ(), you could also have testPrimeQNegativeNumbers(), testPrimeQZero(), testPrimeQNaN(), etc.
It is also possible to write one test for more than one subroutine. This might make sense in the case of polymorphism, such as if you have boolean primeQ(int realInt) and boolean primeQ(GaussianInteger gauInt).
I can attest that in such a case, NetBeans with JUnit would generate two separate tests. You may or may not like such granularity.
This seems as good as any a point to express my opinion on how to test for thrown exceptions. Do you use the modern approach of annotations, or the dated try-catch with fail way?
I would call the latter approach, which I prefer, “classic” rather than “dated.” But if the test suite is granular enough, I can acknowledge that annotations might be the best way to go.
For example, let’s say you have one division function but two tests for that division function. One test works with normal nonzero divisors, and the other test tries to divide a number by zero.
@Test(expected = Exception.class)
public void testDividingByZero() {
System.out.println("Testing division by zero.");
QuadraticRing OQi7 = new QuadraticRing(-7);
QuadraticInteger dividend = new QuadraticInteger(-3, 2, OQi7);
QuadraticInteger divisor = new QuadraticInteger(0, 0, OQi7);
QuadraticInteger result = dividend.dividedBy(divisor);
}If I’ve done this correctly, the test should pass as long as the division function throws any exception at all.
But I think the old try-catch with fail enables you to be both more specific and more general. What if in the division by zero example you want the division function to throw either an ArithmeticException or an IllegalArgumentException to pass, but any other exception fails the test?
There’s probably a way to do that with annotations. But since I prefer to not get too granular, try-catch with fail is easier. The following example would be an excerpt from a longer test:
dividend = new QuadraticInteger(-3, 2, OQi7);
divisor = new QuadraticInteger(0, 0, OQi7);
try {
result = dividend.dividedBy(divisor);
fail("Division by 0 did not trigger any exception.");
} catch (ArithmeticException ae) {
System.out.println("Division by 0 correctly triggered ArithmeticException.");
} catch (IllegalArgumentException iae) {
System.out.println("Division by 0 correctly triggered IllegalArgumentException.");
} catch (Exception e) {
fail("Division by 0 triggered the wrong exception," + e.getMessage());
}The order of the catches matters because both ArithmeticException and IllegalArgumentException are subclasses of Exception (by way of RuntimeException). So if the catch (Exception e) block preceded the other two catch blocks, the test would fail even if the correct exception was triggered.
Actually… the IDE will probably let you know if you have your catches in the wrong order. This is really only a problem for those who still insist on writing their source code in a separate editor rather than the IDE’s editor.
In test-driven development, the try-catch with fail example would express that either ArithmeticException or IllegalArgumentException are valid exceptions to throw in the case of attempting to divide by zero, but Exception or any of its other subclasses would be wrong.
Exception is too vague, even if the attached message is very detailed, and PrinterException seems irrelevant somehow.
So the try-catch with fail in the example is specifically for trying to divide −3 + 2√−7 by 0. It seems unlikely that this would be the one case for which the implementation works as expected but fails for all others.
It would of course be impossible to test this for every number in that domain, since it’s an infinite domain. And it would take too much time to test by every number that can be represented by our implementation of QuadraticInteger, because that’s a finite set but large enough to tie up the computer for too long a stretch.
But it’s certainly practical to put the try-catch in a loop in order to test division by zero with, say, a hundred different numbers. That shouldn’t take the computer more than a few seconds, and it would give us greater confidence in a passing test.
On my journey from unit testing skeptic to believer, I’ve become aware of some of the benefits of unit testing.
All the code examples below come from my Java program to draw diagrams of prime numbers in “imaginary” quadratic integer rings, which is available on GitHub.
Because at first I wasn’t clear on the nuts and bolts of unit testing, it made sense for me to start out by having NetBeans automatically generate the tests for my review, rather than trying to write the tests from scratch.
The first class I needed to test was NumberTheoreticFunctionsCalculator, which mostly operates only on purely “real” integers, but is necessary for the other classes in the project.
So I opened the project in NetBeans and put the cursor on this line:
public class NumberTheoreticFunctionsCalculator {A lightbulb icon replaced the line number. Hovering the mouse over that lightbulb brought up the hints “Create Subclass” and “Create Test Class”.
Clicking on the test class hint brings up more specific hints: my NetBeans installation came pre-loaded with JUnit 4, Selenium (not sure which version) and TestNG (not sure on the version of that one either).
I selected JUnit. NetBeans got to work, creating a package for the test classes, some “boilerplate” and tests with presumed fails. The “boilerplate” included the following:
import org.junit.After;
import org.junit.AfterClass;
import org.junit.Before;
import org.junit.BeforeClass;
import org.junit.Test;
import static org.junit.Assert.*;Behind the scenes NetBeans also takes care of connecting the appropriate JAR files on my system (it would be an understandable misapprehension to think that the IDE will actually go to junit.org to download the relevant package).
I wish I could remember what I did next. Did I run the auto-generated tests right away, or did I spruce them up a little bit first? By default each auto-generated test includes the line
fail("The test case is a prototype.");This might seem like an obvious thing, but I speak from experience: if your test keeps failing after you’ve made several changes to it but you can’t see why, check to see if you have deleted the fail line.
Remember also that the tests are not guaranteed to run in any particular order. I half suspect the frameworks make sure tests don’t run in source code order.
I had read that a few times, but it wasn’t until I had a test fail because of an erroneous test order assumption that this point about test order was etched on my mind.
You will make mistakes when writing the tests, and get false passes and false fails. I know I have (well, in my case I’m only aware of false fails). In the case of NumberTheoreticFunctionsCalculator, all the fails I had were due to mistakes in the tests rather than mistakes in the class being tested.
I had already slowly and carefully tested NumberTheoreticFunctionsCalculator by typing numbers at the command line and noting the answers. With ImaginaryQuadraticRing, the biggest problem I had was writing correct tests for toString(), toHTMLString() and toTeXString().
But the class that was sorely in need of automated testing was ImaginaryQuadraticInteger, because I’m just too slow at mental arithmetic with complex numbers and I didn’t want to abuse Wolfram Alpha with too many queries like 3 * (1/2 + sqrt(-15)/2), 4 * (1/2 + sqrt(-15)/2), 5 * (1/2 + sqrt(-15)/2), etc.
The most obvious benefit of unit testing is ensuring that the algorithm is correct. This is certainly a requirement of a program that does something mathematical.
Unit testing also helps ensure the validity of the object-oriented design. Writing tests made me take a closer look at ImaginaryQuadraticRing and it made me realize I needed to add the function boolean hasHalfIntegers(). I probably wouldn’t have done that if I didn’t do unit testing.
That’s for the principle of data encapsulation. In RingWindowDisplay, I frequently refer to the protected boolean instance field d1mod4. A purist would say that field needs to be private.
But I think, rightly or perhaps wrongly, that RingWindowDisplay using the getter method hasHalfIntegers() instead of accessing d1mod4 directly would incur a performance penalty, resulting in a slower drawing of diagrams (and I still want to improve the performance of the program drawing the Eisenstein primes diagram at 2 pixels per unit interval).
So I choose to rationalize that particular design decision for now. But if in the future I need to write classes that import the imaginaryquadraticinteger package, those classes won’t be able to access d1mod4 directly, so a getter function is essential.
Technically a class and its test class are in the same package. At least that’s how it looks to me in NetBeans with JUnit. Even so, I advise you to pretend that the tests can’t access the protected fields of the classes being tested. I don’t always follow that advice myself, but I do try to go by it most of the time.
And just now I realized that most fields in ImaginaryQuadraticRing need to be final. Upon constructing ImaginaryQuadraticRing(-3), for example,d1mod4 should be set to true and there is no reason for that to ever change.
There is also the question of whether or not you need to override equals() and hashCode(). I had wondered about that and read a little about it. What I read left me with the impression that overriding those methods is a major hassle best avoided.
But unit testing made me face the fact that, at least for ImaginaryQuadraticInteger, it is necessary to override equals(), and therefore also hashCode().
By the way, the IDE, at least NetBeans, is very helpful in writing equals() and hashCode() overrides.
In some cases, unit testing might prompt you to write more constructors. In the case of ImaginaryQuadraticInteger, I decided to add a constructor without a denominator parameter.
So for example, to use ImaginaryQuadraticInteger only for Gaussian integers, instead of having to write something like
ImaginaryQuadraticInteger gaussInt = new ImaginaryQuadraticInteger(a, b, ringGaussian, 1);over and over again you can write
ImaginaryQuadraticInteger gaussInt = new ImaginaryQuadraticInteger(a, b, ringGaussian);Not having to constantly write , 1 might not seem like much, but repeated often enough might get on your nerves.
Of course in such a scenario, you might decide you’d rather subclass ImaginaryQuadraticInteger as GaussianInteger, and then you can just have a 2-argument constructor.
Unit testing can also help you better understand the limitations of your program. With ImaginaryQuadraticInteger, I made a conscious decision to use the primitive int data type to hold the real part and the imaginary part multiple, rather than BigInteger.
A number like 32,768 + 32,768i, for example, does not seem terribly large. But its norm is 2,147,483,648, which is just large enough to overflow the int data type and cause ImaginaryQuadraticInteger.norm() to be wrong.
I had been aware of this limitation from early on. But since I have yet to add diagram dragging capability to the program, I didn’t feel any urgency yet to address this limitation.
Much more seriously, however, the arithmetic functions can cause overflows with numbers closer to 0. This gave me quite a bit of trouble for unit testing ImaginaryQuadraticInteger.divides().
Eventually I added to ImaginaryQuadraticIntegerTest.setUpClass() the ability to determine what is the largest integer that can be used for the real and imaginary parts without causing overflow problems and tests to fail.
For example, in one run of the test suite, setUpClass() pseudorandomly came up with the ring of algebraic integers of Q(√−6,151) for the tests and determined that 295 could be used safely as a real part or imaginary part multiple.
Then it came up with the numbers 172 − 82i, 172 − 82(√−2), 345/2 − 163(√−3)/2, 345/2 − 163(√−7)/2 and 345/2 − 163(√−6,151)/2 for some of the tests. That last number has a norm of 76,252, which is small enough not to worry us about overflows.
I also wrote some basic overflow detection into the arithmetic functions of ImaginaryQuadraticInteger, but I have yet to write tests to show that they correctly throw ArithmeticException when overflows occur.
When you make changes to the program, if you already have a test suite, it is much easier to confirm that the program still works correctly.
In my mathematical diagram program, I have a function that tests whether a given “simple” integer is prime. In my original implementation, the isPrime() function would actually call primeFactors() and use that to determine if the number is prime.
In hindsight, that was a terrible idea. But at the default magnification of the diagrams (40 pixels per unit interval), the inefficiency made no appreciable difference. But to zoom out the Eisenstein primes diagram to 2 pixels per unit interval, the program would take almost 20 seconds.
Twenty seconds would have been acceptable, perhaps even amazing, to Gotthold Eisenstein in the 1840s, but not so much to me today. I added some time benchmarking println() statements in RingWindowDisplay, but after a short while I realized that the real problem was in NumberTheoreticFunctionsCalculator.isPrime().
That function only needs to find a least prime factor, not look at a complete factorization. You can look at a number like 48,015 and immediately tell that it is not prime, but to actually factor it in full would you take you a bit longer.
The same goes for a computer, though of course the computer does it much quicker. Still, a slightly inefficient subroutine repeated enough times can cause a major inefficiency.
So my improved algorithm for isPrime() should work faster and still work correctly. But in the process of typing the new and improved version of the function, I could make some small but crucial mistake that messes it up.
Thanks to having NumberTheoreticFunctionsCalculatorTest ready, checking that my improved isPrime() function works correctly was a simple matter of running the test.
One nice, unsung feature of testing frameworks is a little bit of benchmarking. For instance, a recent run of testIsPrime() on my computer took 0.162 seconds.
I’ve learned that it’s important to have println statements in your tests. But not too many of them, they can really slow things down. A good baseline is that each test should have a println identifying what is being tested.
I know JUnit in NetBeans automatically generates those identification println statements, and I imagine other combinations of testing frameworks and IDEs also do so.
But remember to include identification println statements in any tests you add besides those that were automatically generated. This is another one of those things that I’ve learned from personal experience.
Knowing what order the tests just ran in does not seem terribly important, and in any case the framework will probably let you know. If your setup and tests generate data besides the results of the assertions, it might be a good idea to have println statements for some of that data.
For instance, NumberTheoreticFunctionsCalculatorTest.setUpClass() generates a list of the positive primes below 1000, so it reports that it has generated a list of 168 primes, the 168th prime being 997.
And randomNegativeSquarefreeNumber() comes up with a pseudorandom negative “squarefree” number (not divisible by any perfect square), so the test for that function reports what pseudorandom number the function delivered.
The assertions in the test are what determines if the test passed or failed, but having the tests give a little bit of information about what is going can be a valuable sanity check.
Also, it helps the test duration feel not too long, as you’re not worried so much that your computer may have crashed.
My next steps in unit testing should probably include unit testing the graphical unit interface (GUI) created by RingWindowDisplay. It can be done and it should be done, but I haven’t yet read up on how to do it.
As for integration testing, I’m simply not at that point yet. Maybe when I write a program that uses a database, or a mobile device’s accelerometer.
In summary, unit testing can definitely help you improve the program you’re testing and overall make you a better programmer.
Whether it can make you a more employable programmer, I can’t comment on, as that touches on factors that have nothing to do with your skill, knowledge or ability to work in a team.
