XHTML Basic Test Suite survey

Hi,

As per my action item from our last meeting [1], I have collected the
data I could find on conformance test suites for XHTML Basic 1.1.

As it happens, XHTML Basic 1.1 doesn't have a test suite, but XHTML MP
1.2 (which is the OMA equivalent of the W3C spec) has one, although it
isn't publicly available yet. Here are the data I collected on it as we
discussed:
 * the test suite is for
XHTML Basic 1.1 / XHTML MP 1.2 [pdf]
 * some description about it is available in the  Enabler Test
Specification (Conformance) for XHTML Mobile Profile (1.2) available at:
http://www.openmobilealliance.org/release_program/docs/ETS/OMA-ETS-XHTMLMP_CON-V1_2-20061130-C.pdf
Unfortunately, the test suite itself isn't publicly available (see
below)
 * Its status in OMA terms is "Candidate Version" which means it hasn't
been approved yet
 * in terms of coverage, there seems to be 56 test cases, but there is
no indication of actual coverage;
 * in particular, while the enabler spec references a set of test
assertions, these test assertions seem to be integrated in the actual
test suites, so I wasn't able to review them
 * in terms of documentation, the enabler spec documents the various
test cases, and their expected output, but doesn't give much details as
to what "running" the test case means, nor how to verify that the
referenced criteria are met; the actual test suite may have more details
 * not having seen the test suite, I'm unable to say whether there is a
test harness; I imagine there would be at least a page with links to the
various test cases
 * As I mentioned before, the test suite isn't available to non-OMA
members at this time; there seems to be some ongoing discussions as to
whether they could be made public or not.
 * In terms of maintenance, problems can be reported to the OMA Problem
Reporting system, but that system is also reserved to OMA members;
 the development of the test suite follows the established OMA process:
http://www.openmobilealliance.org/release_program/index.html
 * the enabler spec has some legalese on the use of the spec itself, but
doesn't describe what licenses run for the test suite, so I couldn't
determine the licenses aspect; given the restriction on the availability
of the test suite itself, I expect the default license to be quite
restrictive, but maybe that's also something on which W3C's ongoing
liaison with OMA could help

Generally speaking, the main problem to evaluate that test suite is of
course its lack of availability; the description of the test cases seem
to show that at least each module of XHTML MP 1.2 has a test.

Dom

1. http://www.w3.org/2007/01/30-mwts-minutes.html#action03

Received on Friday, 2 February 2007 14:35:00 UTC