mobileOK Basic Tests 1.0 Reference Implementation, Design and Implementation Notes (Draft)</title1> </head> <body> <h1>mobileOK Basic Tests 1.0 Reference Implementation, Design and Implementation Notes (Draft)</h1> <h2>Table of Contents</h2> <ul> <li><a href="#overview" accesskey="1">1. Overview</a></li> <li><a href="#design" accesskey="1">2. Design</a></li> <li><a href="#usage" accesskey="1">3. Usage</a></li> <li><a href="#implementation" accesskey="1">4. Implementation Notes</a></li> <li><a href="#automatedtests" accesskey="1">5. Automated Tests</a></li> <li><a href="#appendices" accesskey="1">6. Appendices</a></li> </ul> <h2 id="overview">1. Overview</h2> <p>This document describes the design and usage of the mobileOK Basic Tests 1.0 reference implementation. It is intended to be useful to developers who wish to integrate the reference implementation into an existing application or infrastructure.</p> <p><a href="http://www.w3.org/TR/mobileOK-basic10-tests/">mobileOK Basic Tests 1.0</a> specifies the essential behavior required of all implementations, but leaves many other decisions up to particular implementations. It allows for implementations to provide additional functionality beyond what is required. So, this document also discusses implementation-specific decisions taken during development. These are intended to be useful to developers who are porting this implementation to another platform, or, developing a new implementation from scratch.</p> <h2 id="design">2. Design Overview</h2> <p>This reference implementation is written in Java and requires <a href="http://java.sun.com/j2se/1.5.0/">Java SE 5.0</a>. It is a pure Java library, rather than an application. It is intended to be embedded in another application, possibly one that provides a GUI or web-based interface to its functionality.</p> <h3>2.1 Architecture</h3> <p>In essence, the library accepts as input a URI, and outputs two sets of information. In the end, the test results are produced, of course. But much more information is also produced, concerning what occurred during testing -- specifically, detailed information about the document under test, its retrieval and parsing, some derived counts and statistics, and information on linked resources.</p> <p>The test results output satisfies the requirements of mobileOK Basic Tests 1.0. It is simply the results of each of test, plus warning and informational messages. It is available via a native Java API (see <code>TestResults</code> below), and also as an XML document which uses the <a href="http://www.w3.org/TR/EARL10-Schema/">Evaluation and Report Language (EARL)</a> schema.</p> <p>The additional output, named the "preprocessor" output, exists to aid extensibility. The reference implementation does a great deal of preprocessing before it begins test execution, so, it makes sense to expose the details of this preprocessing for other applications to use as well, even though it is not directly related to the test results.</p> <p><img src="architecture-diagram.png" alt="Architecture Diagram" height="658" width="731"/></p> <h3>2.2 Design of public API</h3> <p><img src="class-overview.png" alt="Class Overview" height="448" width="663"/></p> <p>The key API classes are:</p> <h4>Tester</h4> <p>This is the top-level abstraction which runs the test. It can be constructed to operate on a file, a URI, or can be given the HTTP response body and headers directly. It exposes a <code>runTests()</code> method that runs everything, or, <code>runTest()</code> which runs just one test.</p> <h4>TestType</h4> <p>Just an enum of all test types.</p> <h4>TestResults, TestResult</h4> <p>Returned by <code>runTests()</code> and <code>runTest()</code> respectively, and encapsulate the results of many tests and one test, respectively. TestResults includes many TestResult instances. These objects include test outcome, warnings, and information.</p> <h4>TestOutcome</h4> <p>Just an enum of <code>PASS</code> and <code>FAIL</code>, the two test outcomes</p> <h4>TestException</h4> <p>Throw if something goes wrong in the tester -- that is, not just that there was an error accessing the document but that something is wrong with the tester.</p> <h2 id="usage">3. Usage</h2> <h3>3.1 Java</h3> <pre> Tester tester = new Tester("http://example.org"); TestResults results = tester.runTests(); TestOutcome outcome = results.getOverallOutcome(); TestResult = results.getResultForTest(TestType.AUTO_REFRESH); ... TestResult oneResult = tester.runTest(TestType.AUTO_REFRESH); </pre> <h3>3.2 Command line</h3> <p>While provided as a library, it is also possible to get quick results on the command line:</p> <pre> java -cp mobileOKBasic-RI.jar org.w3c.mwi.mobileok.basic.Tester http://example.org </pre> <h2 id="implementation">4. Implementation Notes</h2> <p>This section discusses implementation decisions taken in design and development that were not dictated by the mobileOK Basic Tests specification.</p> <h3>4.1 Preprocessing Results</h3> <p>The most noticeable feature of this implementation is that it exposes a large amount of information about the results of accessing and preprocessing the URI under test. It is exposed as a DOM from <code>TestResults.getPreprocessorDocument()</code> which may be serialized and used by any external tool. The Appendices describe the format of this DOM in detail. This data structure contains the following information:</p> <p><strong>TODO</strong></p> <h3>4.2 Test Results</h3> <p>Test results are available from the method <code>Tester.runTests()</code>, which returns a <code>TestResults</code> object. The same information is available as a DOM from <code>TestResults.getTestResultsDocument()</code>, in a format described in the Appendices. The format uses the EARL RDF vocabulary to describe the outcome. Test results are provided in this form as well to ease integration with external tools. The test results data structures include the following information:</p> <ul> <li>Overall outcome (<code>PASS</code> or <code>FAIL</code>)</li> <li>Global warning messages</li> <li>Global informational messages</li> <li>For each test: <ul> <li>Test outcome (<code>PASS</code> or <code>FAIL</code>)</li> <li>Error message if any</li> <li>Warning messages</li> <li>Informational messages</li> </ul> </li> </ul> <h3>4.3 Notes to extenders</h3> <p>This section contains notes that may be useful to developers who wish to build on this implementation to construct additional tests of mobile content beyond what is defined in mobileOK Basic.</p> <p><strong>TODO</strong></p> <h3>4.4 Malformed markup and tidying</h3> <p>It is expected that the most common cause of test failures is malformed markup. This will cause the VALID_MARKUP test to fail of course, and thus the entire test suite.</p> <p>However, many tests must have a valid DOM to run and produce any useful information. It is of course desirable to provide as much information as possible to the person running the tests rather than completely fail on the common case of malformed markup.</p> <p>So, in the event that the input document cannot be parsed into a DOM, this implementation will attempt to "tidy" the document and then re-run all tests. This does not change the fact that the overall test suite result is FAIL; however, this does potentially allow some tests to run and provide some useful feedback to the developer.</p> <h3>4.5 Third-party packages</h3> <ul> <li><a href="http://xerces.apache.org/xerces2-j/">Xerces 2.9</a></li> <li><a href="http://jakarta.apache.org/commons/codec/">Jakarta Commons Codec 1.3</li> <li><a href="http://jakarta.apache.org/commons/httpclient/">Jakarta Commons HttpClient 3.1</a></li> <li><a href="http://jakarta.apache.org/commons/logging/">Jakarta Commons Logging 1.1</a></li> <li><a href="http://home.ccil.org/~cowan/XML/tagsoup/">TagSoup 1.1</a></li> <li><a href="http://hul.harvard.edu/jhove/">JHOVE 1.1</a></li> <li><a href="http://www.w3.org/Style/CSS/SAC/">SAC (Simple API for CSS) 1.3</a></li> <li><a href="http://junit.org/">JUnit 4.3</a></li> <li><a href="http://tomcat.apache.org/">Tomcat 6</a></li> </ul> <h2 id="automatedtests">5. Automated Tests</h2> <p>Like any good software library, this reference implementation includes automated tests. All tests are implemented using the standard JUnit framework. In addition to some unit tests of common code, this also includes a series of tests consisting of documents and associated HTTP headers plus expected output documents. This makes it easy to run the implementation on many example resources and verify that the implementation outputs expected test results.</p> <p>Test cases may be found in subdirectories of <code>test/testcases</code>. Each subdirectory contains:</p> <ul> <li>The test resource, called <code>index.html</code></li> <li>Any other resources which the test resource links to, like stylesheets or images</li> <li>For each file above, an additional file whose name is suffixed with ".headers". This contains a series of "Name: Value" lines representing the HTTP headers that should be returned in the response containing a document.</li> </ul> <p>The test harness starts an embedded Tomcat server to serve these resources, and employs a special servlet that reads the *.headers files and sets HTTP response headers appropriately. In this way, the implementation can be tested against a real HTTP server.</p> <h2 id="appendices">6. Appendices</h2> <h3>6.1 Intermediate results</h3> <h4>6.1.1 Schema</h4> <h4>6.1.2 Example</h4> <p><strong>TODO</strong></p> <h3>6.2 Test results schema</h3> <h4>6.2.1 Schema</h4> <h4>6.2.2 Example</h4> <p><strong>TODO</strong></p> </body> </html>