blob: 42f29a6573dbbd496058ec10d940315ca91d8278 [file] [log] [blame]
<!doctype html public "-//w3c//dtd html 4.0 transitional//en">
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta name="GENERATOR" content="Microsoft FrontPage 4.0">
<meta name="Author" content="Erich Gamma, Kent Beck, and David Saff">
<title>JUnit 4.6</title>
<b><font color="#00CC00">J</font><font color="#FF0000">U</font><font color="#000000">nit
<br>Brought to you by <a href="">Kent Beck</a>, Erich
Gamma, and <a href="">David Saff</a>.
<br>FAQ edited by <a href="">Mike Clark</a>. Web mastering by Erik
<br>(see also <a href=""></a>)
<hr WIDTH="100%">
<br>6 April 2009
<p>JUnit is a simple framework to write repeatable tests. It is an instance
of the xUnit architecture for unit testing frameworks.
<a href="#Summary of">Summary of Changes</a></li>
<a href="#Contents">Contents</a></li>
<a href="#Installation">Installation</a></li>
<a href="#Getting">Getting Started</a></li>
<a href="#Documentation">Documentation</a></li>
<a href="#Known Defects">Known Defects</a></li>
<a NAME="Summary of">
<h2>Summary of Changes in version 4.6</h2>
<p>JUnit now includes a new experimental Core, <code>MaxCore</code>. <code>MaxCore</code>
remembers the results of previous test runs in order to run new
tests out of order. <code>MaxCore</code> prefers new tests to old tests, fast
tests to slow tests, and recently failing tests to tests that last
failed long ago. There's currently not a standard UI for running
<code>MaxCore</code> included in JUnit, but there is a UI included in the JUnit
Max Eclipse plug-in at:</p>
<pre><code>public static class TwoUnEqualTests {
public void slow() throws InterruptedException {
public void fast() {
public void rememberOldRuns() {
File maxFile = new File("history.max");
MaxCore firstMax = MaxCore.storedLocally(maxFile);;
MaxCore useHistory= MaxCore.storedLocally(maxFile);
List&lt;Failure&gt; failures=
assertEquals("fast", failures.get(0).getDescription().getMethodName());
assertEquals("slow", failures.get(1).getDescription().getMethodName());
<h3>Test scheduling strategies</h3>
<p><code>JUnitCore</code> now includes an experimental method that allows you to
specify a model of the <code>Computer</code> that runs your tests. Currently,
the only built-in Computers are the default, serial runner, and two
runners provided in the <code>ParallelRunner</code> class:
<code>ParallelRunner.classes()</code>, which runs classes in parallel, and
<code>ParallelRunner.methods()</code>, which runs classes and methods in parallel.</p>
<p>This feature is currently less stable than MaxCore, and may be
merged with MaxCore in some way in the future.</p>
<pre><code>public static class Example {
@Test public void one() throws InterruptedException {
@Test public void two() throws InterruptedException {
@Test public void testsRunInParallel() {
long start= System.currentTimeMillis();
Result result= JUnitCore.runClasses(ParallelComputer.methods(),
long end= System.currentTimeMillis();
assertThat(end - start, betweenInclusive(1000, 1500));
<h3>Comparing double arrays</h3>
<p>Arrays of doubles can be compared, using a delta allowance for equality:</p>
public void doubleArraysAreEqual() {
assertArrayEquals(new double[] {1.0, 2.0}, new double[] {1.0, 2.0}, 0.01);
<h3><code>Filter.matchDescription</code> API</h3>
<p>Since 4.0, it has been possible to run a single method using the <code>Request.method</code>
API. In 4.6, the filter that implements this is exposed as <code>Filter.matchDescription</code>.</p>
<li><p>A couple classes and packages that once had empty javadoc have been
<li><p>Added how to run JUnit from the command line to the cookbook.</p></li>
<li><p> now contains build.xml</p></li>
<h3>Bug fixes</h3>
<li>Fixed overly permissive @DataPoint processing (2191102)</li>
<li>Fixed bug in test counting after an ignored method (2106324)</li>
<h2>Summary of Changes in version 4.5</h2>
<li>We are releasing <code>junit-4.6.jar</code>, which contains all the classes
necessary to run JUnit, and <code>junit-dep-4.6.jar</code>, which leaves out
hamcrest classes, for developers who already use hamcrest outside of
<h3>Basic JUnit operation</h3>
<li><p>JUnitCore now more often exits with the correct exit code (0 for
success, 1 for failure)</p></li>
<li><p>Badly formed test classes (exceptions in constructors, classes
without tests, multiple constructors, Suite without @SuiteClasses)
produce more helpful error messages</p></li>
<li><p>Test classes whose only test methods are inherited from superclasses
now run.</p></li>
<li><p>Optimization to annotation processing can cut JUnit overhead by more than half
on large test classes, especially when using Theories. [Bug 1796847]</p></li>
<li><p>A failing assumption in a constructor ignores the class</p></li>
<li><p>Correct results when comparing the string "null" with potentially
null values. [Bug 1857283]</p></li>
<li><p>Annotating a class with <code>@RunWith(JUnit4.class)</code> will always invoke the
default JUnit 4 runner in the current version of JUnit. This default changed
from <code>JUnit4ClassRunner</code> in 4.4 to <code>BlockJUnit4ClassRunner</code> in 4.5 (see below),
and may change again.</p></li>
<li><p><code>BlockJUnit4Runner</code> is a new implementation of the standard JUnit 4
test class functionality. In contrast to <code>JUnit4ClassRunner</code> (the old
<li><p><code>BlockJUnit4Runner</code> has a much simpler implementation based on
Statements, allowing new operations to be inserted into the
appropriate point in the execution flow.</p></li>
<li><p><code>BlockJUnit4Runner</code> is published, and extension and reuse are
encouraged, whereas <code>JUnit4ClassRunner</code> was in an internal package,
and is now deprecated.</p></li>
<li><p><code>ParentRunner</code> is a base class for runners that iterate over
a list of "children", each an object representing a test or suite to run.
<code>ParentRunner</code> provides filtering, sorting, <code>@BeforeClass</code>, <code>@AfterClass</code>,
and method validation to subclasses.</p></li>
<li><p><code>TestClass</code> wraps a class to be run, providing efficient, repeated access
to all methods with a given annotation.</p></li>
<li><p>The new <code>RunnerBuilder</code> API allows extending the behavior of
Suite-like custom runners.</p></li>
<li><p><code>AssumptionViolatedException.toString()</code> is more informative</p></li>
<h3>Extra Runners</h3>
<li><p><code>Parameterized.eachOne()</code> has been removed</p></li>
<li><p>New runner <code>Enclosed</code> runs all static inner classes of an outer class.</p></li>
<li><p><code>@Before</code> and <code>@After</code> methods are run before and after each set of attempted parameters
on a Theory, and each set of parameters is run on a new instance of the test class.</p></li>
<li><p>Exposed API's <code>ParameterSignature.getType()</code> and <code>ParameterSignature.getAnnotations()</code></p></li>
<li><p>An array of data points can be introduced by a field or method
marked with the new annotation <code>@DataPoints</code></p></li>
<li><p>The Theories custom runner has been refactored to make it faster and
easier to extend</p></li>
<li><p>Source has been split into directories <code>src/main/java</code> and
<code>src/test/java</code>, making it easier to exclude tests from builds, and
making JUnit more maven-friendly</p></li>
<li><p>Test classes in <code>org.junit.tests</code> have been organized into
subpackages, hopefully making finding tests easier.</p></li>
<li><p><code>ResultMatchers</code> has more informative descriptions.</p></li>
<li><p><code>TestSystem</code> allows testing return codes and other system-level interactions.</p></li>
<h2>Summary of Changes in version 4.4</h2>
<p>JUnit is designed to efficiently capture developers' intentions about
their code, and quickly check their code matches those intentions.
Over the last year, we've been talking about what things developers
would like to say about their code that have been difficult in the
past, and how we can make them easier.</p>
<p>Two years ago, Joe Walnes built a <a href="">new assertion mechanism</a> on top of what was
then <a href="">JMock 1</a>. The method name was <code>assertThat</code>, and the syntax looked like this:</p>
<pre><code>assertThat(x, is(3));
assertThat(x, is(not(4)));
assertThat(responseString, either(containsString("color")).or(containsString("colour")));
assertThat(myList, hasItem("3"));
<p>More generally:</p>
<pre><code>assertThat([value], [matcher statement]);
<p>Advantages of this assertion syntax include:</p>
<li><p>More readable and typeable: this syntax allows you to think in terms of subject, verb, object
(assert "x is 3") rathern than <code>assertEquals</code>, which uses verb, object, subject (assert "equals 3 x")</p></li>
<li><p>Combinations: any matcher statement <code>s</code> can be negated (<code>not(s)</code>), combined (<code>either(s).or(t)</code>),
mapped to a collection (<code>each(s)</code>), or used in custom combinations (<code>afterFiveSeconds(s)</code>)</p></li>
<li><p>Readable failure messages. Compare</p>
<pre><code>assertTrue(responseString.contains("color") || responseString.contains("colour"));
// ==&gt; failure message:
// java.lang.AssertionError:
assertThat(responseString, anyOf(containsString("color"), containsString("colour")));
// ==&gt; failure message:
// java.lang.AssertionError:
// Expected: (a string containing "color" or a string containing "colour")
// got: "Please choose a font"
<li><p>Custom Matchers. By implementing the <code>Matcher</code> interface yourself, you can get all of the
above benefits for your own custom assertions.</p></li>
<li><p>For a more thorough description of these points, see <a href="">Joe Walnes's
original post</a>.:</p></li>
<p>We have decided to include this API directly in JUnit.
It's an extensible and readable syntax, and because it enables
new features, like <a href="#assumptions">assumptions</a> and <a href="#theories">theories</a>.</p>
<p>Some notes:</p>
<li>The old assert methods are never, ever, going away. <br />
Developers may continue using the old <code>assertEquals</code>, <code>assertTrue</code>, and
so on.</li>
<li><p>The second parameter of an <code>assertThat</code> statement is a <code>Matcher</code>.
We include the Matchers we want as static imports, like this:</p>
<pre><code>import static;
<pre><code>import static org.hamcrest.CoreMatchers.*;
<li><p>Manually importing <code>Matcher</code> methods can be frustrating. [Eclipse
3.3][] includes the ability to
"Favorite" classes to import static methods from, which makes it easier
(Search for "Favorites" in the Preferences dialog).
We expect that support for static imports will improve in all Java IDEs in the future.</p></li>
<li><p>To allow compatibility with a wide variety of possible matchers,
we have decided to include the classes from hamcrest-core,
from the <a href="">Hamcrest</a> project. This is the first time that
third-party classes have been included in JUnit. </p></li>
<li><p>To allow developers to maintain full control of the classpath contents, the JUnit distribution also provides an unbundled junit-dep jar,
ie without hamcrest-core classes included. This is intended for situations when using other libraries that also depend on hamcrest-core, to
avoid classloading conflicts or issues. Developers using junit-dep should ensure a compatible version of hamcrest-core jar (ie 1.1+) is present in the classpath.</p></li>
<li><p>JUnit currently ships with a few matchers, defined in
<code>org.hamcrest.CoreMatchers</code> and <code>org.junit.matchers.JUnitMatchers</code>. <br />
To use many, many more, consider downloading the <a href="">full hamcrest package</a>.</p></li>
<li><p>JUnit contains special support for comparing string and array
values, giving specific information on how they differ. This is not
yet available using the <code>assertThat</code> syntax, but we hope to bring
the two assert methods into closer alignment in future releases.</p></li>
<p><a name="assumptions" />
Ideally, the developer writing a test has control of all of the forces that might cause a test to fail.
If this isn't immediately possible, making dependencies explicit can often improve a design. <br />
For example, if a test fails when run in a different locale than the developer intended,
it can be fixed by explicitly passing a locale to the domain code.</p>
<p>However, sometimes this is not desirable or possible. <br />
It's good to be able to run a test against the code as it is currently written,
implicit assumptions and all, or to write a test that exposes a known bug.
For these situations, JUnit now includes the ability to express "assumptions":</p>
<pre><code>import static org.junit.Assume.*
@Test public void filenameIncludesUsername() {
assumeThat(File.separatorChar, is('/'));
assertThat(new User("optimus").configFileName(), is("configfiles/optimus.cfg"));
@Test public void correctBehaviorWhenFilenameIsNull() {
assumeTrue(bugFixed("13356")); // bugFixed is not included in JUnit
assertThat(parse(null), is(new NullDocument()));
<p>With this beta release, a failed assumption will lead to the test being marked as passing,
regardless of what the code below the assumption may assert.
In the future, this may change, and a failed assumption may lead to the test being ignored:
however, third-party runners do not currently allow this option.</p>
<p>We have included <code>assumeTrue</code> for convenience, but thanks to the
inclusion of Hamcrest, we do not need to create <code>assumeEquals</code>,
<code>assumeSame</code>, and other analogues to the <code>assert*</code> methods. All of
those functionalities are subsumed in assumeThat, with the appropriate
<p>A failing assumption in a <code>@Before</code> or <code>@BeforeClass</code> method will have the same effect
as a failing assumption in each <code>@Test</code> method of the class.</p>
<p><a name="theories" />
More flexible and expressive assertions, combined with the ability to
state assumptions clearly, lead to a new kind of statement of intent,
which we call a "Theory". A test captures the intended behavior in
one particular scenario. A theory allows a developer to be
as precise as desired about the behavior of the code in possibly
infinite numbers of possible scenarios. For example:</p>
public class UserTest {
@DataPoint public static String GOOD_USERNAME = "optimus";
@DataPoint public static String USERNAME_WITH_SLASH = "optimus/prime";
@Theory public void filenameIncludesUsername(String username) {
assumeThat(username, not(containsString("/")));
assertThat(new User(username).configFileName(), containsString(username));
<p>This makes it clear that the user's filename should be included in the
config file name, only if it doesn't contain a slash. Another test
or theory might define what happens when a username does contain a slash.</p>
<p><code>UserTest</code> will attempt to run <code>filenameIncludesUsername</code> on
every compatible <code>DataPoint</code> defined in the class. If any of the
assumptions fail, the data point is silently ignored. If all of the
assumptions pass, but an assertion fails, the test fails.</p>
<p>The support for Theories has been absorbed from the <a href="">Popper</a>
project, and <a href="">more complete documentation</a> can be found
<p>Defining general statements in this way can jog the developer's memory
about other potential data points and tests, also allows <a href="">automated
tools</a> to <a href="">search</a> for new, unexpected data
points that expose bugs.</p>
<h3>Other changes</h3>
<p>This release contains other bug fixes and new features. Among them:</p>
<li><p>Annotated descriptions</p>
<p>Runner UIs, Filters, and Sorters operate on Descriptions of test
methods and test classes. These Descriptions now include the
annotations on the original Java source element, allowing for richer
display of test results, and easier development of annotation-based
<li><p>Bug fix (1715326): assertEquals now compares all Numbers using their
native implementation of <code>equals</code>. This assertion, which passed in
4.3, will now fail:</p>
<p>assertEquals(new Integer(1), new Long(1));</p>
<p>Non-integer Numbers (Floats, Doubles, BigDecimals, etc),
which were compared incorrectly in 4.3, are now fixed.</p></li>
<li><p><code>assertEquals(long, long)</code> and <code>assertEquals(double, double)</code> have
been re-introduced to the <code>Assert</code> class, to take advantage of
Java's native widening conversions. Therefore, this still passes:</p>
<p>assertEquals(1, 1L);</p></li>
<li><p>The default runner for JUnit 4 test classes has been refactored.
The old version was named <code>TestClassRunner</code>, and the new is named
<code>JUnit4ClassRunner</code>. Likewise, <code>OldTestClassRunner</code> is now
<code>JUnit3ClassRunner</code>. The new design allows variations in running
individual test classes to be expressed with fewer custom classes.
For a good example, see the source to
<li><p>The rules for determining which runner is applied by default to a
test class have been simplified:</p>
<li><p>If the class has a <code>@RunWith</code> annotation, the annotated runner
class is used.</p></li>
<li><p>If the class can be run with the JUnit 3 test runner (it
subclasses <code>TestCase</code>, or contains a <code>public static Test suite()</code>
method), JUnit38ClassRunner is used.</p></li>
<li><p>Otherwise, JUnit4ClassRunner is used.</p></li>
<p>This default guess can always be overridden by an explicit
<code>@RunWith(JUnit4ClassRunner.class)</code> or
<code>@RunWith(JUnit38ClassRunner.class)</code> annotation.</p>
<p>The old class names <code>TestClassRunner</code> and <code>OldTestClassRunner</code>
remain as deprecated.</p></li>
<li><p>Bug fix (1739095): Filters and Sorters work correctly on test
classes that contain a <code>suite</code> method like:</p>
<p>public static junit.framework.Test suite() {
return new JUnit4TestAdapter(MyTest.class);
<li><p>Bug fix (1745048): @After methods are now correctly called
after a test method times out.</p></li>
<a NAME="Summary of"></a>Summary of Changes in version 4.3.1</h2>
<li>Bug fix: 4.3 introduced a
<a href="">bug</a>
that caused a NullPointerException
when comparing a null reference to a non-null reference in <tt>assertEquals</tt>.
This has been fixed.
<li>Bug fix: The binary jar for 4.3 <a href="">accidentally</a> included the tests and sample code,
which are now removed for a smaller download, but, as always, available from the
full zip.
<a NAME="Summary of"></a>Summary of Changes with version 4.3</h2>
<li>Changes in array equality. Using <tt>assertEquals</tt> to compare array contents is now deprecated.
In the future, <tt>assertEquals</tt> will revert to its pre-4.0 meaning of comparing objects based on
Java's <tt>Object.equals</tt> semantics. To compare array contents, use the new, more reliable
<tt>Assert.assertArrayEquals</tt> methods.
<li>The <tt>@Ignore</tt> annotation can now be applied to classes, to ignore the entire class, instead of
individual methods.
<li>Originally, developers who wanted to use a static <tt>suite()</tt> method from JUnit 3.x with a JUnit 4.x
runner had to annotate the class with <tt>@RunWith(AllTests.class)</tt>. In the common case, this requirement
has been removed. However, when such a class is wrapped with a JUnit4TestAdapter (which we believe is rare), the
results may not be as expected.
<li>Improved error messages for array comparison("arrays first differed at element [1][0]")
<li>Bug fix: Inaccessible base class is caught at test construction time.
<li>Bug fix: Circular suites are caught at test construction time.
<li>Bug fix: Test constructors that throw exceptions are reported correctly.
<li><b>For committers and extenders</b>
<li>Sources now are in a separate "src" directory (this means a big break in the CVS history)
<li>Improved documentation in <tt>Request</tt>, <tt>RunWith</tt>
<a NAME="Summary of"></a>Summary of Changes with version 4.2</h2>
<li>Bug fix: Inaccessible base class is caught at test construction time.
<li>Bug fix: Circular suites are caught at test construction time.
<li>Improved error messages for array comparison("arrays first differed at element [1][0]")
<li>Test constructors that throw exceptions are reported correctly.
<a NAME="Summary of"></a>Summary of Changes with version 4.1</h2>
<li>Bug fix: listeners now get a correct test running time, rather than always being told 0 secs.
<li>The @RunWith annotation is now inherited by subclasses:
all subclasses of an abstract test class will be run by the same runner.
<li>The build script fails if the JUnit unit tests fail
<li>The faq has been updated
<li>Javadoc has been improved, with more internal links, and package descriptions added (Thanks, Matthias Schmidt!)
<li>An acknowledgements.txt file has been created to credit outside contributions
<li>The <tt>Enclosed</tt> runner, which runs all of the static inner classes of a given class, has been added
to <tt>org.junit.runners</tt>.
<h2>Summary of Changes with version 4.0</h2>
The architecture of JUnit 4.0 is a substantial departure from that of earlier releases.
Instead of
tagging test classes by subclassing junit.framework.TestCase and tagging test methods by
starting their name with "test", you now tag test methods with the @Test annotation.
<a NAME="Contents"></a>Contents of the Release</h2>
<td>this file</td>
<td>a jar file with the JUnit framework, bundled with the hamcrest-core-1.1 dependency.</td>
<td>a jar file with the JUnit framework, unbundled from any external dependencies.
Choosing to use this jar developers will need to also provide in the classpath a compatible version of external dependencies (ie hamcrest-core-1.1+)</td>
<td>a jar file with the source code of the JUnit framework</td>
<td>the source code of the basic JUnit annotations and classes</td>
<td><tt>&nbsp;&nbsp;&nbsp; samples</tt></td>
<td>sample test cases</td>
<td><tt>&nbsp;&nbsp;&nbsp; tests</tt></td>
<td>test cases for JUnit itself</td>
<td>javadoc generated documentation</td>
<td>documentation and articles</td>
<a NAME="Installation"></a>Installation</h2>
Below are the installation steps for installing JUnit:
unzip the file</li>
add<i> </i><b>junit-4.6.jar</b> to the CLASSPATH. For example:
<tt> set classpath=%classpath%;INSTALL_DIR\junit-4.6.jar;INSTALL_DIR</tt></li>
test the installation by running <tt>java org.junit.runner.JUnitCore org.junit.tests.AllTests</tt></li>
<br><b><font color="#FF0000">Notice</font></b>: that the tests are not
contained in the junit-4.6.jar but in the installation directory directly.
Therefore make sure that the installation directory is on the class path
<b><font color="#FF0000">Important</font></b>: don't install junit-4.6.jar
into the extension directory of your JDK installation. If you do so the
test class on the files system will not be found.
<a NAME="Getting"></a>Getting Started</h2>
To get started with unit testing and JUnit read the article:
<a href="doc/cookbook/cookbook.htm">JUnit Cookbook</a>.
<br>This article describes basic test writing using JUnit 4.
<p>You find additional samples in the org.junit.samples package:
<li> - some simple test cases</li>
<li> - test cases for java.util.Vector</li>
<a NAME="Documentation"></a>Documentation</h2>
<blockquote><a href="doc/cookbook/cookbook.htm">JUnit Cookbook</a>
<br>&nbsp;&nbsp;&nbsp; A cookbook for implementing tests with JUnit.
<br><a href="javadoc/index.html">Javadoc</a>
<br>&nbsp;&nbsp;&nbsp; API documentation generated with javadoc.
<br><a href="doc/faq/faq.htm">Frequently asked questions</a>
<br>&nbsp;&nbsp;&nbsp; Some frequently asked questions about using JUnit.
<br><a href="cpl-v10.html">License</a>
<br>&nbsp;&nbsp;&nbsp; The terms of the common public license used for JUnit.<br>
The following documents still describe JUnit 3.8.
<br><a href="doc/testinfected/testing.htm">Test Infected - Programmers
Love Writing Tests</a>
<br>&nbsp;&nbsp;&nbsp; An article demonstrating the development process
with JUnit.
<br><a href="doc/cookstour/cookstour.htm">JUnit - A cooks tour</a>
<hr WIDTH="100%">
<!--webbot bot="HTMLMarkup" startspan --><a href=""><IMG
width="88" height="31" border="0" alt="SourceForge Logo"></a><!--webbot
bot="HTMLMarkup" endspan -->