# How to Write a QA Test

### Author: Steven F. Harris

### Date: September 28, 1999

### Introduction

This document explains in step by step fashion how to write a Jini QA test. A Jini QA test is a class that implements the org.apache.river.tck.harness.Test interface and is thus able to run inside of the TCK (Technology Compatibility Kit) test harness. In general, a Jini QA test will test either a service (such as Lookup Discovery Service) or a utility (such as Client Lookup Manager or Join Manager).

It is recommended that each test be written to test one or a related set of assertions. These assertions will either be specification based or implementation based. In the former case the test is asserting something about conformance to a written Jini specification. In the latter case, the assertion relates to an implementation specific aspect of the entity under test. The requirements for writing these two types of tests differ slightly and that difference is explained later on in this document.

- ### QA Test directory structure

All QA Test directories reside under `/vob/qa/src/org/apache/river/qa`. A pictorial view of this structure follows:

                            /vob/org/apache/river/qa
                                     |
      -------------------------------------------------------------------------
      |      |         |         |      |                   |
      +      +         +         +      +                   +
    docs  resource  examples  harness  spec                impl
                                        |                   |
                                        `- lookupservice    `- reggie
                                        |                   |
                                        `- javaspace        `- outrigger
                                        |                   |
                                        `- txnmanager       `- mahalo
                                        |                   |
                                        `- discoveryservice `- fiddler
                                        |                   |
                                        `- renewalservice   `- norm
                                        |                   |
                                        `- eventmailbox     `- mercury
                                        |                   |
                                        `- lookupmanager    `- clientlookupmanager
                                        |                   |
                                        `- renewalmanager   `- leaserenewalmanager
                                        |                   |
                                        `- joinmanager      `- joinmanager

There are two types of tests. Those that test an assertion or set of related assertions based on requirements as laid out in a technical specification and those that test behavior based upon some aspect of the implementation. The former reside in the subdirectory `/vob/org/apache/river/qa/spec` directory and the latter reside in the subdirectory `/vob/org/apache/river/qa/impl`. There is a one-to-one correspondence between the specification subdirectories and the implementation subdirectories. That is the pair (lookupservice, reggie) represents the subdirectories that contain code that tests conformance to The Jini Lookup Service Specification and tests that are only concerned with testing implementation dependent aspects of Sun's Lookup Service called `reggie`. It is possible, in theory, that a specification subdirectory could be associated with multiple implementation subdirectories if Sun or some third party decided to produce alternate implementations of the same specification.

If you are writing tests for a heretofore untested service or utility, then you should create the appropriate directories as necessary.

- ### Setting up the Environment

Make certain that you have the most recent build of the necessary code by executing the following from the command line :

1. `cd /vob/tools/src`
2. `gmake clean`
3. `gmake`
4. `gmake jars`
5. `cd /vob/jive/src`
6. `gmake clean`
7. `gmake`
8. `gmake jars`
9. `cd /vob/tck/src`
10. `gmake clean`
11. `gmake`
12. `gmake jars`
13. `cd /vob/qa/src`
14. `gmake clean`
15. `gmake`
16. `gmake jars`

Insure that whatever compile process you use includes the following in the CLASSPATH :

1. `/vob/jive/lib/qa1.jar`

- ### Writing a Test Step by Step

1. Copy the /vob/qa/src/org/apache/river/qa/resource/QATestTemplate.txt file into the package directory of the test you are writing.
2. Rename QATestTemplate.txt to the name of the test class you are writing.
3. Bring the file up in your favorite editor.
4. Search for code buoys and fill in the information as requested.

A code buoy is a tag in the code that acts as a placeholder for information (code or comment) that needs to be supplied in the process of writing a Jini QA test. Each code buoy starts with the string "<XXX" and ends with the text "XXX>". What appears in between is a word or phrase indicating the nature of the required information. For example the place holder for the package name is expressed as <XXXpackage_nameXXX>. Each test requires the following information :

1. <XXXpackage_nameXXX>
2. <XXXclass_descriptionXXX>
3. <XXXauthorXXX>
4. <XXXclass_nameXXX>
5. <XXXdescription_of_the_test_goes_here_XXX>
6. <XXXcategory_in_lower_caseXXX>
7. <XXXa_generous_description_of_test_objectiveXXX>
8. <XXXsuccess_messageXXX>

5. Find the `setup()` method and include any required setup code.
6. Find the `run()` method and add code to test the required assertion(s).
7. Find the `tearDown()` method and add code to clean up test state.
8. Congratulations, you have just written a Jini QA test!

Test writers are encouraged to log all important actions in a test as an aid to diagnosing logic faults during automated test runs. Please be generous in logging important actions by using the `log` PrintWriter. The `log` PrintWriter is a public instance variable inherited from the parent QATest class.

- ### The `QATestUtil` class

All tests are derived from the `org.apache.river.qa.harness.QATest` class. This parent class provides a protected instance member named `util` which is an instance of the class `org.apache.river.qa.harness.QATestUtil`. `QATestUtil` provides common methods which all Jini QA tests should use as needed to achieve their objectives.

`QATestUtil` provides methods that can be categorized into three different areas. A summary of these methods are listed in the tables below. For more information see the QATestUtil java doc.

Capturing Test Parameters
`boolean` | `getBooleanArgValue(java.lang.String keyword, boolean defaultValue)` Retrieve an boolean argument from command line or failing that, the the test's properties file.
`double` | `getDoubleArgValue(java.lang.String keyword, double defaultValue)` Retrieve an double argument from command line or failing that, the the test's properties file.
`java.io.File` | `getFileArgValue(java.lang.String keyword, java.io.File defaultValue)` Retrieve an File argument from command line or failing that, the the test's properties file.
`float` | `getFloatArgValue(java.lang.String keyword, float defaultValue)` Retrieve an float argument from command line or failing that, the the test's properties file.
`int` | `getIntArgValue(java.lang.String keyword, int defaultValue)` Retrieve an integer argument from command line or failing that, the the test's properties file.
`long` | `getLongArgValue(java.lang.String keyword, long defaultValue)` Retrieve an long argument from command line or failing that, the the test's properties file.
`java.lang.String` | `getStringArgValue(java.lang.String keyword, java.lang.String defaultValue)` Retrieve an String argument from command line or failing that, the the test's properties file.

Starting/Querying Services
`net.jini.core.lookup.ServiceID` | `getServiceID(java.lang.Object serviceProxy)` Return the service id of the service that publishes the supplied proxy.
`java.lang.Object` | `requestService(java.lang.Class interfaceClass)` Request a service via its interface class.
`void` | `stopAllServices()` Stop all the services started up via the requestService() method
`void` | `stopService(java.lang.Object serviceProxy)` Stop a service that has been obtained via the requestService() method.

Modifying Lookup Services
`void` | `setLookupGroups(net.jini.core.lookup.ServiceRegistrar serviceProxy, java.lang.String[] groups)` Change the set of groups to which a lookup belongs

File System Support
`static java.io.File` | `createUniqueDirectory(java.lang.String prefix, java.lang.String suffix, java.lang.String path)` Creates a directory (folder) that is guaranteed to be unique on the file system and returns it as a java.io.File object.
`static java.lang.String` | `createUniqueFileName(java.lang.String prefix, java.lang.String suffix, java.lang.String path)` Create a filename that is guaranteed to be unique on the file system.

- ### Sample Program

The best way to see how the above utility methods are used is to look at the example SampleUtilityTest.java.

- ### Additional Information

There are special test parameters that can be used to control various aspects of the testing system. For a detailed description these and other features of the QA Test Methodology see The Conformance Methodology Architecture Document.