Preliminary report on XHTML MP test suite

Hi,

As I announced last week, the OMA test suite for XHTML MP has been made
available publicly on the OMA web site:
http://testfest.openmobilealliance.org/XHTML_Content/index.html

I've started to review the tests available there; my review isn't
completed yet, but I thought I would share some preliminary
observations:
* the accompanying documentation hasn't be made public yet (i.e. I get
prompted for a username/password for
http://testfest.openmobilealliance.org/docs/XHTML_Tests_V1_0_2006_10_20.doc ) ; I'll report it to our OMA liaison

* the relationships between the 3 test suites (3a, b and c) isn't
extremely clear to me yet, but they don't seem to be 3 versions of the
same test suite, although I do see some overlap between 3a and 3b

* quite a few tests in 3a apply to validating user agents; as I don't
think any user agent on the market claims to be validating, these tests
would be candidate for removals should we try to repackage them

* the tests aren't available for download as a package

* some tests aren't valid or well-formed where it's not clear they were
meant to be so; a few examples:
 -
http://testfest.openmobilealliance.org/XHTML_Content/xhtml_Structure/case_sense_ns2.xhtml  and http://testfest.openmobilealliance.org/XHTML_Content/xhtml_Structure/case_sense_ns3.xhtml don't have the XHTML namespace declared (this isn't mandatory, but triggers an error in Firefox)
 -
http://testfest.openmobilealliance.org/XHTML_Content/xhtml_UA_conformance/content_type/content_html.xhtml and http://testfest.openmobilealliance.org/XHTML_Content/xhtml_UA_conformance/content_type/content_xhtml.xhtml aren't well-formed due to a non-escaped ampersand; also the actual tests are hosted on a different server (testsuite.nokia-boston.com)

I stumbled upon quite a few of others that aren't well-formed; I guess
running a full validation of the tests would be useful

* licensing-wise, the license seems to allow for derivative works: "This
License determines your rights with respect to the Software. You may:
[...] Create derivative works of the Software (“Derivative Works”),
provided that You hereby grant the Open Mobile Alliance a non-exclusive,
worldwide, payment-free, royalty-free copyright and patent license to
reproduce, prepare derivative works of, publicly display, publicly
perform, distribute and sublicense the Derivative Works, in source code
and object code form."
http://testfest.openmobilealliance.org/home_pc.asp
which I think would allow us to re-use some of the tests in our own
package; I guess that would need more careful examination by W3C legal
team should we choose to do so

* there are quite a few tests, but no clear indication of coverage again

* some tests links to dead-end ("post results" links don't lead to
anything useful at this time)

Dom

Received on Monday, 26 February 2007 09:53:41 UTC