annotate doc/testing.html @ 54576:aa626cbadf1b

8222550: runtime/MemberName/ times out Reviewed-by: coleenp, dholmes
author stefank
date Wed, 17 Apr 2019 07:41:12 +0200
parents 5ae4d3f46537
rev   line source
ihse@45227 1 <!DOCTYPE html>
jiefu@54473 2 <html xmlns="" lang="" xml:lang="">
ihse@44511 3 <head>
jiefu@54473 4 <meta charset="utf-8" />
jiefu@54473 5 <meta name="generator" content="pandoc" />
jiefu@54473 6 <meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes" />
mr@50885 7 <title>Testing the JDK</title>
jiefu@54473 8 <style type="text/css">
jiefu@54473 9 code{white-space: pre-wrap;}
jiefu@54473 10 span.smallcaps{font-variant: small-caps;}
jiefu@54473 11 span.underline{text-decoration: underline;}
jiefu@54473 12 div.column{display: inline-block; vertical-align: top; width: 50%;}
jiefu@54473 13 </style>
jiefu@54473 14 <link rel="stylesheet" href="../make/data/docs-resources/resources/jdk-default.css" />
ihse@45227 15 <!--[if lt IE 9]>
ihse@45227 16 <script src="//"></script>
ihse@45227 17 <![endif]-->
ihse@44511 18 <style type="text/css">pre, code, tt { color: #1d6ae5; }</style>
ihse@44511 19 </head>
ihse@44511 20 <body>
jiefu@54473 21 <header id="title-block-header">
mr@50885 22 <h1 class="title">Testing the JDK</h1>
ihse@45227 23 </header>
ihse@45227 24 <nav id="TOC">
ihse@44511 25 <ul>
ihse@52342 26 <li><a href="#using-make-test-the-run-test-framework">Using &quot;make test&quot; (the run-test framework)</a><ul>
ihse@45618 27 <li><a href="#configuration">Configuration</a></li>
ihse@45618 28 </ul></li>
ihse@44511 29 <li><a href="#test-selection">Test selection</a><ul>
ihse@45618 30 <li><a href="#jtreg">JTReg</a></li>
ihse@44511 31 <li><a href="#gtest">Gtest</a></li>
redestad@52595 32 <li><a href="#microbenchmarks">Microbenchmarks</a></li>
ihse@52342 33 <li><a href="#special-tests">Special tests</a></li>
ihse@44511 34 </ul></li>
ihse@44511 35 <li><a href="#test-results-and-summary">Test results and summary</a></li>
ihse@44511 36 <li><a href="#test-suite-control">Test suite control</a><ul>
ihse@53464 37 <li><a href="#general-keywords-test_opts">General keywords (TEST_OPTS)</a></li>
ihse@45618 38 <li><a href="#jtreg-keywords">JTReg keywords</a></li>
ihse@44511 39 <li><a href="#gtest-keywords">Gtest keywords</a></li>
redestad@52595 40 <li><a href="#microbenchmark-keywords">Microbenchmark keywords</a></li>
ihse@44511 41 </ul></li>
jiefu@54473 42 <li><a href="#notes-for-specific-tests">Notes for Specific Tests</a><ul>
jiefu@54473 43 <li><a href="#docker-tests">Docker Tests</a></li>
erikj@54570 44 <li><a href="#non-us-locale">Non-US locale</a></li>
jiefu@54473 45 </ul></li>
ihse@44511 46 </ul>
ihse@45227 47 </nav>
ihse@52342 48 <h2 id="using-make-test-the-run-test-framework">Using &quot;make test&quot; (the run-test framework)</h2>
mr@50885 49 <p>This new way of running tests is developer-centric. It assumes that you have built a JDK locally and want to test it. Running common test targets is simple, and more complex ad-hoc combination of tests is possible. The user interface is forgiving, and clearly report errors it cannot resolve.</p>
ihse@52342 50 <p>The main target <code>test</code> uses the jdk-image as the tested product. There is also an alternate target <code>exploded-test</code> that uses the exploded image instead. Not all tests will run successfully on the exploded image, but using this target can greatly improve rebuild times for certain workflows.</p>
dholmes@54268 51 <p>Previously, <code>make test</code> was used to invoke an old system for running tests, and <code>make run-test</code> was used for the new test framework. For backward compatibility with scripts and muscle memory, <code>run-test</code> (and variants like <code>exploded-run-test</code> or <code>run-test-tier1</code>) are kept as aliases.</p>
ihse@44511 52 <p>Some example command-lines:</p>
ihse@52342 53 <pre><code>$ make test-tier1
ihse@52342 54 $ make test-jdk_lang JTREG=&quot;JOBS=8&quot;
ihse@52342 55 $ make test TEST=jdk_lang
ihse@52342 56 $ make test-only TEST=&quot;gtest:LogTagSet gtest:LogTagSetDescriptions&quot; GTEST=&quot;REPEAT=-1&quot;
ihse@52342 57 $ make test TEST=&quot;hotspot:hotspot_gc&quot; JTREG=&quot;JOBS=1;TIMEOUT=8;VM_OPTIONS=-XshowSettings -Xlog:gc+ref=debug&quot;
ihse@52342 58 $ make test TEST=&quot;jtreg:test/hotspot:hotspot_gc test/hotspot/jtreg/native_sanity/;
redestad@52595 59 $ make test TEST=&quot;micro:java.lang.reflect&quot; MICRO=&quot;FORK=1;WARMUP_ITER=2&quot;
ihse@52342 60 $ make exploded-test TEST=tier2</code></pre>
ihse@45618 61 <h3 id="configuration">Configuration</h3>
ihse@51644 62 <p>To be able to run JTReg tests, <code>configure</code> needs to know where to find the JTReg test framework. If it is not picked up automatically by configure, use the <code>--with-jtreg=&lt;path to jtreg home&gt;</code> option to point to the JTReg framework. Note that this option should point to the JTReg home, i.e. the top directory, containing <code>lib/jtreg.jar</code> etc. (An alternative is to set the <code>JT_HOME</code> environment variable to point to the JTReg home before running <code>configure</code>.)</p>
redestad@52595 63 <p>To be able to run microbenchmarks, <code>configure</code> needs to know where to find the JMH dependency. Use <code>--with-jmh=&lt;path to JMH jars&gt;</code> to point to a directory containing the core JMH and transitive dependencies. The recommended dependencies can be retrieved by running <code>sh make/devkit/</code>, after which <code>--with-jmh=build/jmh/jars</code> should work.</p>
ihse@44511 64 <h2 id="test-selection">Test selection</h2>
ihse@52342 65 <p>All functionality is available using the <code>test</code> make target. In this use case, the test or tests to be executed is controlled using the <code>TEST</code> variable. To speed up subsequent test runs with no source code changes, <code>test-only</code> can be used instead, which do not depend on the source and test image build.</p>
ihse@52342 66 <p>For some common top-level tests, direct make targets have been generated. This includes all JTReg test groups, the hotspot gtest, and custom tests (if present). This means that <code>make test-tier1</code> is equivalent to <code>make test TEST=&quot;tier1&quot;</code>, but the latter is more tab-completion friendly. For more complex test runs, the <code>test TEST=&quot;x&quot;</code> solution needs to be used.</p>
ihse@50267 67 <p>The test specifications given in <code>TEST</code> is parsed into fully qualified test descriptors, which clearly and unambigously show which tests will be run. As an example, <code>:tier1</code> will expand to <code>jtreg:$(TOPDIR)/test/hotspot/jtreg:tier1 jtreg:$(TOPDIR)/test/jdk:tier1 jtreg:$(TOPDIR)/test/langtools:tier1 jtreg:$(TOPDIR)/test/nashorn:tier1 jtreg:$(TOPDIR)/test/jaxp:tier1</code>. You can always submit a list of fully qualified test descriptors in the <code>TEST</code> variable if you want to shortcut the parser.</p>
ihse@45618 68 <h3 id="jtreg">JTReg</h3>
ihse@50267 69 <p>JTReg tests can be selected either by picking a JTReg test group, or a selection of files or directories containing JTReg tests.</p>
mr@50885 70 <p>JTReg test groups can be specified either without a test root, e.g. <code>:tier1</code> (or <code>tier1</code>, the initial colon is optional), or with, e.g. <code>hotspot:tier1</code>, <code>test/jdk:jdk_util</code> or <code>$(TOPDIR)/test/hotspot/jtreg:hotspot_all</code>. The test root can be specified either as an absolute path, or a path relative to the JDK top directory, or the <code>test</code> directory. For simplicity, the hotspot JTReg test root, which really is <code>hotspot/jtreg</code> can be abbreviated as just <code>hotspot</code>.</p>
ihse@50267 71 <p>When specified without a test root, all matching groups from all test roots will be added. Otherwise, only the group from the specified test root will be added.</p>
mr@50885 72 <p>Individual JTReg tests or directories containing JTReg tests can also be specified, like <code>test/hotspot/jtreg/native_sanity/</code> or <code>hotspot/jtreg/native_sanity</code>. Just like for test root selection, you can either specify an absolute path (which can even point to JTReg tests outside the source tree), or a path relative to either the JDK top directory or the <code>test</code> directory. <code>hotspot</code> can be used as an alias for <code>hotspot/jtreg</code> here as well.</p>
ihse@50267 73 <p>As long as the test groups or test paths can be uniquely resolved, you do not need to enter the <code>jtreg:</code> prefix. If this is not possible, or if you want to use a fully qualified test descriptor, add <code>jtreg:</code>, e.g. <code>jtreg:test/hotspot/jtreg/native_sanity</code>.</p>
ihse@44511 74 <h3 id="gtest">Gtest</h3>
ihse@44511 75 <p>Since the Hotspot Gtest suite is so quick, the default is to run all tests. This is specified by just <code>gtest</code>, or as a fully qualified test descriptor <code>gtest:all</code>.</p>
ihse@44511 76 <p>If you want, you can single out an individual test or a group of tests, for instance <code>gtest:LogDecorations</code> or <code>gtest:LogDecorations.level_test_vm</code>. This can be particularly useful if you want to run a shaky test repeatedly.</p>
ihse@51644 77 <p>For Gtest, there is a separate test suite for each JVM variant. The JVM variant is defined by adding <code>/&lt;variant&gt;</code> to the test descriptor, e.g. <code>gtest:Log/client</code>. If you specify no variant, gtest will run once for each JVM variant present (e.g. server, client). So if you only have the server JVM present, then <code>gtest:all</code> will be equivalent to <code>gtest:all/server</code>.</p>
redestad@52595 78 <h3 id="microbenchmarks">Microbenchmarks</h3>
redestad@52595 79 <p>Which microbenchmarks to run is selected using a regular expression following the <code>micro:</code> test descriptor, e.g., <code>micro:java.lang.reflect</code>. This delegates the test selection to JMH, meaning package name, class name and even benchmark method names can be used to select tests.</p>
redestad@52595 80 <p>Using special characters like <code>|</code> in the regular expression is possible, but needs to be escaped multiple times: <code>micro:ArrayCopy\\\\\|reflect</code>.</p>
ihse@52342 81 <h3 id="special-tests">Special tests</h3>
iignatyev@52396 82 <p>A handful of odd tests that are not covered by any other testing framework are accessible using the <code>special:</code> test descriptor. Currently, this includes <code>failure-handler</code> and <code>make</code>.</p>
ihse@52342 83 <ul>
ihse@52342 84 <li><p>Failure handler testing is run using <code>special:failure-handler</code> or just <code>failure-handler</code> as test descriptor.</p></li>
ihse@52342 85 <li><p>Tests for the build system, including both makefiles and related functionality, is run using <code>special:make</code> or just <code>make</code> as test descriptor. This is equivalent to <code>special:make:all</code>.</p>
ihse@52342 86 <p>A specific make test can be run by supplying it as argument, e.g. <code>special:make:idea</code>. As a special syntax, this can also be expressed as <code>make-idea</code>, which allows for command lines as <code>make test-make-idea</code>.</p></li>
ihse@52342 87 </ul>
ihse@44511 88 <h2 id="test-results-and-summary">Test results and summary</h2>
ihse@44511 89 <p>At the end of the test run, a summary of all tests run will be presented. This will have a consistent look, regardless of what test suites were used. This is a sample summary:</p>
ihse@44511 90 <pre><code>==============================
ihse@44511 91 Test summary
ihse@44511 92 ==============================
ihse@44511 94 &gt;&gt; jtreg:jdk/test:tier1 1867 1865 2 0 &lt;&lt;
ihse@44511 95 jtreg:langtools/test:tier1 4711 4711 0 0
ihse@44511 96 jtreg:nashorn/test:tier1 133 133 0 0
ihse@44511 97 ==============================
ihse@44511 98 TEST FAILURE</code></pre>
ihse@44511 99 <p>Tests where the number of TOTAL tests does not equal the number of PASSed tests will be considered a test failure. These are marked with the <code>&gt;&gt; ... &lt;&lt;</code> marker for easy identification.</p>
ihse@44511 100 <p>The classification of non-passed tests differs a bit between test suites. In the summary, ERROR is used as a catch-all for tests that neither passed nor are classified as failed by the framework. This might indicate test framework error, timeout or other problems.</p>
ihse@52342 101 <p>In case of test failures, <code>make test</code> will exit with a non-zero exit value.</p>
ihse@51644 102 <p>All tests have their result stored in <code>build/$BUILD/test-results/$TEST_ID</code>, where TEST_ID is a path-safe conversion from the fully qualified test descriptor, e.g. for <code>jtreg:jdk/test:tier1</code> the TEST_ID is <code>jtreg_jdk_test_tier1</code>. This path is also printed in the log at the end of the test run.</p>
ihse@44511 103 <p>Additional work data is stored in <code>build/$BUILD/test-support/$TEST_ID</code>. For some frameworks, this directory might contain information that is useful in determining the cause of a failed test.</p>
ihse@44511 104 <h2 id="test-suite-control">Test suite control</h2>
ihse@44511 105 <p>It is possible to control various aspects of the test suites using make control variables.</p>
ihse@45618 106 <p>These variables use a keyword=value approach to allow multiple values to be set. So, for instance, <code>JTREG=&quot;JOBS=1;TIMEOUT=8&quot;</code> will set the JTReg concurrency level to 1 and the timeout factor to 8. This is equivalent to setting <code>JTREG_JOBS=1 JTREG_TIMEOUT=8</code>, but using the keyword format means that the <code>JTREG</code> variable is parsed and verified for correctness, so <code>JTREG=&quot;TMIEOUT=8&quot;</code> would give an error, while <code>JTREG_TMIEOUT=8</code> would just pass unnoticed.</p>
iignatyev@49293 107 <p>To separate multiple keyword=value pairs, use <code>;</code> (semicolon). Since the shell normally eats <code>;</code>, the recommended usage is to write the assignment inside qoutes, e.g. <code>JTREG=&quot;...;...&quot;</code>. This will also make sure spaces are preserved, as in <code>JTREG=&quot;VM_OPTIONS=-XshowSettings -Xlog:gc+ref=debug&quot;</code>.</p>
ihse@51644 108 <p>(Other ways are possible, e.g. using backslash: <code>JTREG=JOBS=1\;TIMEOUT=8</code>. Also, as a special technique, the string <code>%20</code> will be replaced with space for certain options, e.g. <code>JTREG=VM_OPTIONS=-XshowSettings%20-Xlog:gc+ref=debug</code>. This can be useful if you have layers of scripts and have trouble getting proper quoting of command line arguments through.)</p>
ihse@44511 109 <p>As far as possible, the names of the keywords have been standardized between test suites.</p>
ihse@53464 110 <h3 id="general-keywords-test_opts">General keywords (TEST_OPTS)</h3>
ihse@53464 111 <p>Some keywords are valid across different test suites. If you want to run tests from multiple test suites, or just don't want to care which test suite specific control variable to use, then you can use the general TEST_OPTS control variable.</p>
ihse@53464 112 <p>There are also some keywords that applies globally to the test runner system, not to any specific test suites. These are also available as TEST_OPTS keywords.</p>
ihse@53464 113 <h4 id="jobs">JOBS</h4>
ihse@53464 114 <p>Currently only applies to JTReg.</p>
ihse@53464 115 <h4 id="timeout_factor">TIMEOUT_FACTOR</h4>
ihse@53464 116 <p>Currently only applies to JTReg.</p>
ihse@53464 117 <h4 id="vm_options">VM_OPTIONS</h4>
ihse@53464 118 <p>Applies to JTReg, GTest and Micro.</p>
ihse@53464 119 <h4 id="java_options">JAVA_OPTIONS</h4>
ihse@53464 120 <p>Applies to JTReg, GTest and Micro.</p>
ihse@53464 121 <h4 id="aot_modules">AOT_MODULES</h4>
ihse@53464 122 <p>Applies to JTReg and GTest.</p>
ihse@53464 123 <h4 id="jcov">JCOV</h4>
ihse@53464 124 <p>This keywords applies globally to the test runner system. If set to <code>true</code>, it enables JCov coverage reporting for all tests run. To be useful, the JDK under test must be run with a JDK built with JCov instrumentation (<code>configure --with-jcov=&lt;path to directory containing lib/jcov.jar&gt;</code>, <code>make jcov-image</code>).</p>
ihse@53464 125 <p>The simplest way to run tests with JCov coverage report is to use the special target <code>jcov-test</code> instead of <code>test</code>, e.g. <code>make jcov-test TEST=jdk_lang</code>. This will make sure the JCov image is built, and that JCov reporting is enabled.</p>
ihse@53464 126 <p>The JCov report is stored in <code>build/$BUILD/test-results/jcov-output</code>.</p>
ihse@53464 127 <p>Please note that running with JCov reporting can be very memory intensive.</p>
ihse@45618 128 <h3 id="jtreg-keywords">JTReg keywords</h3>
ihse@53464 129 <h4 id="jobs-1">JOBS</h4>
ihse@44511 130 <p>The test concurrency (<code>-concurrency</code>).</p>
aoqi@53996 131 <p>Defaults to TEST_JOBS (if set by <code>--with-test-jobs=</code>), otherwise it defaults to JOBS, except for Hotspot, where the default is <em>number of CPU cores/2</em> (for sparc, if more than 16 cpus, then <em>number of CPU cores/5</em>, otherwise <em>number of CPU cores/4</em>), but never more than <em>memory size in GB/2</em>.</p>
ihse@53464 132 <h4 id="timeout_factor-1">TIMEOUT_FACTOR</h4>
ihse@44511 133 <p>The timeout factor (<code>-timeoutFactor</code>).</p>
ihse@44511 134 <p>Defaults to 4.</p>
ihse@44511 135 <h4 id="test_mode">TEST_MODE</h4>
ihse@44511 136 <p>The test mode (<code>-agentvm</code>, <code>-samevm</code> or <code>-othervm</code>).</p>
ihse@44511 137 <p>Defaults to <code>-agentvm</code>.</p>
ihse@44511 138 <h4 id="assert">ASSERT</h4>
ihse@44511 139 <p>Enable asserts (<code>-ea -esa</code>, or none).</p>
ihse@44511 140 <p>Set to <code>true</code> or <code>false</code>. If true, adds <code>-ea -esa</code>. Defaults to true, except for hotspot.</p>
ihse@44511 141 <h4 id="verbose">VERBOSE</h4>
ihse@44511 142 <p>The verbosity level (<code>-verbose</code>).</p>
ihse@44511 143 <p>Defaults to <code>fail,error,summary</code>.</p>
ihse@44511 144 <h4 id="retain">RETAIN</h4>
ihse@44511 145 <p>What test data to retain (<code>-retain</code>).</p>
ihse@44511 146 <p>Defaults to <code>fail,error</code>.</p>
ihse@44511 147 <h4 id="max_mem">MAX_MEM</h4>
ihse@44511 148 <p>Limit memory consumption (<code>-Xmx</code> and <code>-vmoption:-Xmx</code>, or none).</p>
ihse@45618 149 <p>Limit memory consumption for JTReg test framework and VM under test. Set to 0 to disable the limits.</p>
ihse@44511 150 <p>Defaults to 512m, except for hotspot, where it defaults to 0 (no limit).</p>
ihse@53464 151 <h4 id="keywords">KEYWORDS</h4>
ihse@53464 152 <p>JTReg kewords sent to JTReg using <code>-k</code>. Please be careful in making sure that spaces and special characters (like <code>!</code>) are properly quoted. To avoid some issues, the special value <code>%20</code> can be used instead of space.</p>
ihse@53464 153 <h4 id="extra_problem_lists">EXTRA_PROBLEM_LISTS</h4>
ihse@53464 154 <p>Use additional problem lists file or files, in addition to the default ProblemList.txt located at the JTReg test roots.</p>
ihse@53464 155 <p>If multiple file names are specified, they should be separated by space (or, to help avoid quoting issues, the special value <code>%20</code>).</p>
ihse@53464 156 <p>The file names should be either absolute, or relative to the JTReg test root of the tests to be run.</p>
ihse@44511 157 <h4 id="options">OPTIONS</h4>
ihse@45618 158 <p>Additional options to the JTReg test framework.</p>
ihse@45618 159 <p>Use <code>JTREG=&quot;OPTIONS=--help all&quot;</code> to see all available JTReg options.</p>
ihse@53464 160 <h4 id="java_options-1">JAVA_OPTIONS</h4>
ihse@45618 161 <p>Additional Java options to JTReg (<code>-javaoption</code>).</p>
ihse@53464 162 <h4 id="vm_options-1">VM_OPTIONS</h4>
ihse@45618 163 <p>Additional VM options to JTReg (<code>-vmoption</code>).</p>
ihse@53464 164 <h4 id="aot_modules-1">AOT_MODULES</h4>
ihse@53464 165 <p>Generate AOT modules before testing for the specified module, or set of modules. If multiple modules are specified, they should be separated by space (or, to help avoid quoting issues, the special value <code>%20</code>).</p>
ihse@44511 166 <h3 id="gtest-keywords">Gtest keywords</h3>
ihse@44511 167 <h4 id="repeat">REPEAT</h4>
ihse@44511 168 <p>The number of times to repeat the tests (<code>--gtest_repeat</code>).</p>
ihse@44511 169 <p>Default is 1. Set to -1 to repeat indefinitely. This can be especially useful combined with <code>OPTIONS=--gtest_break_on_failure</code> to reproduce an intermittent problem.</p>
ihse@44511 170 <h4 id="options-1">OPTIONS</h4>
ihse@44511 171 <p>Additional options to the Gtest test framework.</p>
ihse@44511 172 <p>Use <code>GTEST=&quot;OPTIONS=--help&quot;</code> to see all available Gtest options.</p>
ihse@53464 173 <h4 id="aot_modules-2">AOT_MODULES</h4>
ihse@53464 174 <p>Generate AOT modules before testing for the specified module, or set of modules. If multiple modules are specified, they should be separated by space (or, to help avoid quoting issues, the special value <code>%20</code>).</p>
redestad@52595 175 <h3 id="microbenchmark-keywords">Microbenchmark keywords</h3>
redestad@52595 176 <h4 id="fork">FORK</h4>
redestad@52595 177 <p>Override the number of benchmark forks to spawn. Same as specifying <code>-f &lt;num&gt;</code>.</p>
redestad@52595 178 <h4 id="iter">ITER</h4>
redestad@52595 179 <p>Number of measurement iterations per fork. Same as specifying <code>-i &lt;num&gt;</code>.</p>
redestad@52595 180 <h4 id="time">TIME</h4>
redestad@52595 181 <p>Amount of time to spend in each measurement iteration, in seconds. Same as specifying <code>-r &lt;num&gt;</code></p>
redestad@52595 182 <h4 id="warmup_iter">WARMUP_ITER</h4>
redestad@52595 183 <p>Number of warmup iterations to run before the measurement phase in each fork. Same as specifying <code>-wi &lt;num&gt;</code>.</p>
redestad@52595 184 <h4 id="warmup_time">WARMUP_TIME</h4>
redestad@52595 185 <p>Amount of time to spend in each warmup iteration. Same as specifying <code>-w &lt;num&gt;</code>.</p>
redestad@52595 186 <h4 id="results_format">RESULTS_FORMAT</h4>
redestad@52595 187 <p>Specify to have the test run save a log of the values. Accepts the same values as <code>-rff</code>, i.e., <code>text</code>, <code>csv</code>, <code>scsv</code>, <code>json</code>, or <code>latex</code>.</p>
ihse@53464 188 <h4 id="vm_options-2">VM_OPTIONS</h4>
redestad@52595 189 <p>Additional VM arguments to provide to forked off VMs. Same as <code>-jvmArgs &lt;args&gt;</code></p>
redestad@52595 190 <h4 id="options-2">OPTIONS</h4>
redestad@52595 191 <p>Additional arguments to send to JMH.</p>
jiefu@54473 192 <h2 id="notes-for-specific-tests">Notes for Specific Tests</h2>
jiefu@54473 193 <h3 id="docker-tests">Docker Tests</h3>
jiefu@54473 194 <p>Docker tests with default parameters may fail on systems with glibc versions not compatible with the one used in the default docker image (e.g., Oracle Linux 7.6 for x86). For example, they pass on Ubuntu 16.04 but fail on Ubuntu 18.04 if run like this on x86:</p>
mseledtsov@54525 195 <pre><code>$ make test TEST=&quot;jtreg:test/hotspot/jtreg/containers/docker&quot;</code></pre>
jiefu@54473 196 <p>To run these tests correctly, additional parameters for the correct docker image are required on Ubuntu 18.04 by using <code>JAVA_OPTIONS</code>.</p>
mseledtsov@54525 197 <pre><code>$ make test TEST=&quot;jtreg:test/hotspot/jtreg/containers/docker&quot; JTREG=&quot; -Djdk.test.docker.image.version=latest&quot;</code></pre>
erikj@54570 198 <h3 id="non-us-locale">Non-US locale</h3>
erikj@54570 199 <p>If your locale is non-US, some tests are likely to fail. To work around this you can set the locale to US. On Unix platforms simply setting <code>LANG=&quot;en_US&quot;</code> in the environment before running tests should work. On Windows, setting <code>JTREG=&quot;VM_OPTIONS=-Duser.language=en;</code> helps for most, but not all test cases. For example:</p>
erikj@54570 200 <pre><code>$ export LANG=&quot;en_US&quot; &amp;&amp; make test TEST=...
erikj@54570 201 $ make test JTREG=&quot;VM_OPTIONS=-Duser.language=en; TEST=...</code></pre>
ihse@44511 202 </body>
ihse@44511 203 </html>