Nearly all tests for Xalan are independent Java classes built into testxsl.jar that can be run either individually on the command line, programmatically from your application or our handy Ant build.xml file, or in batches from XSLTestHarness. There really isn't any magic to them: you can just set your classpath and execute java.exe to run them; some Tests and Testlets currently provide defaults for their inputs, so you can run them without any setup at all. However we have provided a couple of more convenient ways to run the most common tests:

If you need to debug into the tests themselves, you may want to use the debug*.bat files

Of course, first Build a fresh copy of testxsl.jar.

cd xml-xalan\test

You can either: use the Ant build.xml script; run a convenience batch file; or execute java.exe yourself.

build conf [Ant-prefixed options]
(runs StylesheetTestletDriver over tests\conf test tree using the default StylesheetTestlet)

build perf [Ant-prefixed options]
(runs StylesheetTestletDriver over tests\perf test tree using the default PerformanceTestlet)

build api -DtestClass=TRAXAPITestClassName [Ant-prefixed options]
(runs TRAX interface tests with Xalan-J 2.x

Alternately: some convenience batch files are provided for the most common tests - these simply turn around and call build.bat/.sh for you.
(Namely conf.bat, perf.bat, and the like)

Alternately: Run java.exe and the desired test class yourself:

Running tests with alternate JAXP parsers: all org.apache.qetest.trax.* tests can be run with Xalan-J 2.x and any JAXP 1.1 compatible parser, like crimson.jar. Be sure to manually set the appropriate system properties to use your parser instead of xerces.jar, which is the default for Xalan-J. Tests will also run on any JAXP 1.1 compatible xslt processor, namely either our default xalan one or our new xsltc one.

Most tests both send basic results to System.out, as well as writing a full set of output results to their logFile, as set from a .properties file or the command line. Generally the output results file is easier to deal with. The basic format is fairly simple and you can certainly read it as-is. Also, many tests send only summary results to the console, but full output to the results file.

To 'pretty-print' results or produce reports, please use the viewResults.xsl stylesheet and associated batch/shell files, like so:
cd \xml-xalan\test

build conf -DoptionName=optionValue -Dqetest.logFile results/MyResults.xml

viewResults.bat results/MyResults.xml results/MyPrettyResults.html [options]

These options are passed to Xalan's command line to transform the xml formatted output results into a nicer-looking html file. The most common option would be -param loggingLevel 99 to get more output shown in the results (higher numbers up to 99 show more details, lower numbers, down to 0, show fewer details).

Alternatively, the tableResults.xsl stylesheet can be used to pretty-print results from the conformance test for multiple TRAX flavours:

cd \xml-xalan\test

build alltest.conf

set RESULTSCANNER=tableResults.xsl

viewResults.bat results-alltest/conf/sax/results.xml results.html [options]

This will generate a pretty-printed HTML table in results.html, listing all flavours as well as the results of individual test cases for each category. The options can be passed to tableResults.xsl to generate conformance reports on previously run tests, or to compare two runs of the conformance test suite.

Possible options that can be passed to tableResults.xsl:

  • -param resultsDir /path/to/result/results-alltest/conf

    Passing the path to the results-alltest/conf directory generates a report based on the results in that directory. Defaults to ./results-alltest/conf

  • -param compareAgainst /path/to/other/results-alltest/conf

    Passing the path to anothe results-alltest/conf directory compares that result against the result pointed to by resultsDir.

  • -param resultsDir results-alltest.xsltc/conf

    Generates a report on the results of an xsltc test (with the alltest.conf.xsltc) target.

Future work includes greatly updated results analysis stylesheets. See FailScanner.xsl and PerfScanner.xsl for ideas. An important design principle in the tests is that at runtime the tests merely output whatever data they can as results are generated; afterwards, we then post-process the results into whatever presentation format is needed, including perhaps re-calculating overall results (example: if we have a list of known fails correlated to JIRA numbers, then a stylesheet could filter out these fails and report them as known bugs instead.

Section needs updating to reflect the fact that while the options the tests see remain as below, the user must now prefix all optionNames with 'qetest.' or 'conf.', etc. when passing the options on the command line or via my.test.properties or test.properties -sc

Most tests can either accept options from a properties file, via:
  TestName -load file.properties

or simply from the command line (which overrides the properties file) like:
  TestName -arg1 value1 -arg2 -arg3 value3

To see all options, call a test with an illegal argument to force it to print out it's .usage(). You may mix setting options from a properties file and from the command line; command line options will take precedence.

For another description of options, see
xml-xalan\test\test.properties,
which describes most of them with comments. Remember that the prefixes 'qetest.', 'conf.' etc. are used by the Ant build.xml file to manage which properties are used for different kinds of tests, and are ripped off before being passed to the Java test script itself. Thus qetest.loggingLevel=99 in the test.properties file becomes just loggingLevel of 99 when passed to the test.

Path-like options set in a properties file generally should use forward slashes (legal in URL's), even on Windows platforms.

Quick list of options

sends test results to an XML-based results file determines how much information is sent to your logFile, 0=very little, 99=lots determines how much information is sent just to the default ConsoleLogger: since often you won't be watching the console as the test is running, you can set this lower than your loggingLevel to speed the tests up a little path to a directory tree of input *.xml/*.xsl files, using your system's separator where all output is sent path to a directory tree of reference output - this tree should be a parallel structure to the inputDir tree only run this single named subdir within inputDir will skip running any specifically named tests; do not use any path elements here which kind/flavor of Processor to test; see ProcessorWrapper.java For StylesheetTestletDriver, use a different class for the testing algorithim (read in a .properties file, that can set any/all of the other opts) This option is automatically used by the Ant build.xml file, so you normally would not specify it yourself.

Note that most options work equivalently with either Xalan-J or Xalan-C tests.

When running tests using Ant, the <xalantest> task actually marshalls various Ant variables and uses the precompiled org.apache.qetest.xsl.XSLTestAntTask to actually execute the test (either in the same JVM or forked depending on ${fork-tests}. It simply strips the appropriate set of prefixes off of required Ant variables and dumps them into an XSLTestAntTask.properties file on disk, which it tells the test executing to -load.

In progress. A few C++ API tests are checked into the xml-xalan/c/Tests repository area already. To execute any set of 'conformance' tests with the Xalan-C processor, we currently use the org.apache.qetest.xsl.XalanCTestlet driver. This is written in Java to take advantage of the framework and results reporting, but basically constructs a command line for each test and then shells out to TestXSLT.exe -in file.xsl... to run the test; it then uses the same validation routines as the Java ConformanceTest.