annotate README @ 405:04bd0247edbb

7901946: javadoc cannot be generated with the latest jdk9-ea build Summary: Only generate the Javadocs for the "safe" parts of API, including jcstress annotations.
author shade
date Mon, 08 May 2017 16:37:41 +0200
rev   line source
shade@350 1 The Java Concurrency Stress tests (jcstress) is an experimental harness and
shade@350 2 a suite of tests to aid the research in the correctness of concurrency support
shade@350 3 in the JVM, class libraries, and hardware.
shade@350 4
shade@350 5 === Quick start
shade@350 6
shade@350 7 You should have Mercurial and Maven installed to check out and build the tests.
shade@350 8 You will need JDK 9+ to compile all the tests. Most tests are runnable on JDK 6+
shade@350 9 afterwards.
shade@350 10
shade@350 11 Check out the jcstress source:
shade@350 12 $ hg clone jcstress
shade@350 13 $ cd jcstress/
shade@350 14
shade@350 15 Build jcstress. Since jcstress is a large project, you may want to build only
shade@350 16 the minimal subset of it:
shade@350 17 $ mvn clean install -pl tests-custom -am
shade@350 18 $ java -jar tests-custom/target/jcstress.jar
shade@350 19
shade@350 20 Run the JAR with -h to get the help.
shade@350 21
shade@350 22 == Caveats
shade@350 23
shade@350 24 *) The harness and tests are heavily experimental, and can change without notice
shade@350 25 as we understand the methodology better, fix issues, and improve the test
shade@350 26 reliability and correctness.
shade@350 27
shade@350 28 *) The project requires JDK 9+ to build and run. It can reference the APIs from
shade@350 29 the future releases, as the jcstress harness will fail gracefully on API
shade@350 30 mismatches, and the mismatched tests will be just skipped.
shade@350 31
shade@350 32 *) Most of the tests are probabilistic, and require substantial time to catch
shade@350 33 all the cases. It is highly recommended to run tests longer to get reliable
shade@350 34 results.
shade@350 35
shade@350 36 *) Most of the tests require at least 2 online CPUs. Low CPU count machines could
shade@350 37 also use these tests, but harness will force yielding there.
shade@350 38
shade@350 39 *) Test failure does not immediately mean the implementation bug. The usual
shade@350 40 suspects are the bugs in test infrastructure, test grading error, bugs in
shade@350 41 hardware, or something else. Share your results, discuss them, we will figure
shade@350 42 out what's wrong. Discuss the result on the relevant mailing lists first.
shade@350 43
shade@350 44 Two usual options are:
shade@350 45 general discussion on concurrency
shade@350 46 to discuss jcstress issues
shade@350 47
shade@350 48 == Understanding the tests and Interpreting the results
shade@350 49
shade@350 50 The tests are similar to what Litmus tests look like, where few threads are
shade@350 51 executing the test concurrently, sometimes rendezvous'ing over the shared state.
shade@350 52 There are multiple state objects generated per each run. Threads then either mutate
shade@350 53 or observe that state object. Test harness is collecting statistics on the observed
shade@350 54 states. In many cases this is enough to catch the reorderings or contract violations
shade@350 55 for concurrent code.
shade@350 56
shade@350 57 The console output can be used to track progress and debugging. Ordinary users should
shade@350 58 use generated HTML report, which has the full interpretation of the results.
shade@350 59
shade@350 60 == Developing tests
shade@350 61
shade@350 62 Please consider contributing the interesting tests back. We follow the OpenJDK policy
shade@350 63 on contributions:
shade@350 64
shade@350 65 If you want to develop a test, you are encouraged to get familiar with existing set of
shade@350 66 tests first. You will have to have a class annotated with jcstress annotations, see the
shade@350 67 harness API. Read up their Javadocs to understand the conditions that are guaranteed for
shade@350 68 those tests. If you need some other test interface/harness support, please don't hesitate
shade@350 69 to raise the issue and describe the scenario you want to test.
shade@350 70
shade@350 71 You are encouraged to provide the thorough explanation why particular test outcome is
shade@350 72 acceptable/forbidden/special. Even though harness will print the debug output into the
shade@350 73 console if no description is given.
shade@350 74
shade@350 75 == Project layout
shade@350 76
shade@350 77 *) jcstress-core: jcstress infrastructure itself. Every jcstress-driven project should
shade@350 78 depend on this module. If you have the standalone jcstress tests, you may depend on this
shade@350 79 module alone.
shade@350 80
shade@350 81 *) jcstress-test-gen: Generator which builds the autogenerated tests in the suite.
shade@350 82
shade@350 83 *) tests-chapter-*: Generated tests for JDK. The build of this module will invoke the
shade@350 84 test generator, and compile the generated tests.
shade@350 85
shade@350 86 *) tests-custom: Custom tests, not coming from the generator. You can use this module to
shade@350 87 add your own custom tests. Use the JAR for this module to only run the custom tests.
shade@350 88
shade@350 89 *) tests-all: Aggregates all the tests. The build of this module will merge the tests available
shade@350 90 in other modules.