How to Write API Tests
How to Write Stylesheet Tests

Use the existing framework! It provides lots of useful functionality, and will help us to maintain your tests.

For example: To write a new TRAX/javax.xml.transform API test:

- open org.apache.qetest.trax.REPLACE_template_for_new_tests.java

- follow directions to rename the file (and put in the correct dom/sax/stream subdir, if needed) and search-and-replace all REPLACE_* tokens

- write one-time-only setup code in testFileInit()

- write a number of testCasen methods. Each one should be independent from the other test cases. Try to test between one and ten or so individual test points (or calls to reporter.check(...) for each testCase method.

- Never use System.out/.err - always use reporter.log*Msg() (to report general messages), or reporter.check() (to validate a specific test point)

This is an important point. Bottlenecking all output from the tests through a Reporter allows us to manage and analyze the results much more easily. Reporters also put all output to both the console and to your logFile.

- Build the tests, including your new one, as described

- Put your test's supporting xml/xsl files in xml-xalan/test/tests/api/trax or subdirectories

- Use xml-xalan\test\traxapitest.bat (and APITest.properties) to run your test! Results will be placed by default into xml-xalan\test\results-api\APITest.xml

The same basic template can be used for other kinds of API tests, with appropriate changes to the package name, etc.

You can pretty-print the results by using the viewResults.xsl stylesheet to turn the XML into an HTML format.

Test cases in the "conf" group will ultimately be submitted to OASIS as part of their vendor-independent project on XSLT/XPath Conformance Testing. For more information about this project, visit http://www.oasis-open.org/committees/xslt/index.html

The OASIS project will combine test cases from different sources and will provide a way to customize the set of tests according to design decisions made for each processor under test. We expect that when the OASIS test system is delivered, it will supplant most or all of the "conf" group provided here. Conformance tests are designed as streamlined "atomic" (or at least "molecular") tests, so there are several hundred of them. Currently, Lotus/IBM is working on submitting a significant body of Conformance tests to OASIS. We also would like to temporarily provide most of the tests here on Apache for Xalan developer's use, while OASIS finalizes it's test suite.

You are invited to submit additional test cases by checking them into the "contrib" area or mailing them to David_Marston@lotus.com. If you want to help with testing Xalan and wish to be assigned to write some cases, send email to David Marston stating your interest. Contributed tests may be sent along to the OASIS conformance project if they test conformance. Guidelines for comments in tests are still evolving as part of the OASIS project, so it may be necessary to add more comments or other annotation after the test is submitted. We hope to have a template for contributing stylesheet tests available soon that will closely match the eventual OASIS format.

The Xalan team will continue to provide test automation, like the StylesheetTestletDriver and various StylesheetTestlet classes, that enables a user to easily run large suites of tests with fully rolled-up reporting and a rich set of options.