view README @ 431:7dce6af27fd5

HashMap failure test
author shade
date Sun, 11 Aug 2019 10:37:40 +0200
line wrap: on
line source
The Java Concurrency Stress tests (jcstress) is an experimental harness and
a suite of tests to aid the research in the correctness of concurrency support
in the JVM, class libraries, and hardware.

=== Quick start

You should have Mercurial and Maven installed to check out and build the tests.
You will need JDK 9+ to compile all the tests. Most tests are runnable on JDK 6+

Check out the jcstress source:
  $ hg clone jcstress
  $ cd jcstress/

Build jcstress. Since jcstress is a large project, you may want to build only
the minimal subset of it:
  $ mvn clean install -pl tests-custom -am
  $ java -jar tests-custom/target/jcstress.jar

Run the JAR with -h to get the help.

== Caveats

 *) The harness and tests are heavily experimental, and can change without notice
    as we understand the methodology better, fix issues, and improve the test
    reliability and correctness.

 *) The project requires JDK 9+ to build and run. It can reference the APIs from
    the future releases, as the jcstress harness will fail gracefully on API
    mismatches, and the mismatched tests will be just skipped.

 *) Most of the tests are probabilistic, and require substantial time to catch
    all the cases. It is highly recommended to run tests longer to get reliable

 *) Most of the tests require at least 2 online CPUs. Low CPU count machines could
    also use these tests, but harness will force yielding there.

 *) Test failure does not immediately mean the implementation bug. The usual
    suspects are the bugs in test infrastructure, test grading error, bugs in
    hardware, or something else. Share your results, discuss them, we will figure
    out what's wrong. Discuss the result on the relevant mailing lists first.

    Two usual options are: general discussion on concurrency to discuss jcstress issues

== Understanding the tests and Interpreting the results

The tests are similar to what Litmus tests look like, where few threads are
executing the test concurrently, sometimes rendezvous'ing over the shared state.
There are multiple state objects generated per each run. Threads then either mutate
or observe that state object. Test harness is collecting statistics on the observed
states. In many cases this is enough to catch the reorderings or contract violations
for concurrent code.

The console output can be used to track progress and debugging. Ordinary users should
use generated HTML report, which has the full interpretation of the results.

== Developing tests

Please consider contributing the interesting tests back. We follow the OpenJDK policy
on contributions:

If you want to develop a test, you are encouraged to get familiar with existing set of
tests first. You will have to have a class annotated with jcstress annotations, see the
harness API. Read up their Javadocs to understand the conditions that are guaranteed for
those tests. If you need some other test interface/harness support, please don't hesitate
to raise the issue and describe the scenario you want to test.

You are encouraged to provide the thorough explanation why particular test outcome is
acceptable/forbidden/special. Even though harness will print the debug output into the
console if no description is given.

== Project layout

 *) jcstress-core: jcstress infrastructure itself. Every jcstress-driven project should
    depend on this module. If you have the standalone jcstress tests, you may depend on this
    module alone.

 *) jcstress-test-gen: Generator which builds the autogenerated tests in the suite.

 *) tests-chapter-*: Generated tests for JDK. The build of this module will invoke the
    test generator, and compile the generated tests.

 *) tests-custom: Custom tests, not coming from the generator. You can use this module to
    add your own custom tests. Use the JAR for this module to only run the custom tests.

 *) tests-all: Aggregates all the tests. The build of this module will merge the tests available
    in other modules.