Test suites for XHTML family modules

DanC raised an interesting issue this week: as we move these small 
modules to Rec, what kind of a test suite can we supply?

I spoke with him about this briefly yesterday, and we explored a few 
options.  Basically, here's the problem:  a small module like xhtml-role 
is really just an attribute and its implementation in DTD and XML 
Schema.  It is not a markup language, and there are no real requirements 
with regard to the behavior of a user agent when the attribute is in use 
(we could argue there are DOM requirements, but we don't actually 
require the DOM).  In fact, these small modules ONLY have meaning when 
used in the context of a markup language.  In that context, it makes 
sense to do a variety of tests that exercise user agents.  But even 
then, if there are no requirements that effect a user agent's behavior 
in the presence of attributes or elements from the module (@role is a 
good example) then it is still really really hard to write valid, 
interesting tests.

So.... here's my thinking.  Tests are a good thing.  More tests are a 
better thing.  Historically we have not provided *any* tests for our 
modules.  That can't be good, and it sure isn't in the spirit of the QA 
requirements that the W3C is trying to enforce these days.

In the case of these small modules, I think it makes sense to *test the 
implementation we supply*.  By this I mean test that the DTD or XML 
Schema modules we supply actually work correctly when used as a 
component of a markup language.  I think this has to be done in 
conjunction with validation technologies and sample documents that 
utilize a simple grammar that includes the module(s) and a minimal .

Obviously, this is a substantial departure from traditional W3C 
testing...  but in this case I think it is the best we can do *and 
require*.  Beyond this, I think that it would be useful in the case of 
@role to supply additional, informative tests, that exercise @role in 
conjunction with ARIA.  I assume they are creating tests already - there 
must be something there we can leverage or do together.  That might help 
ensure that the components we expect to be available in a DOM tree are 
there when @role is used in a user agent.  We could even do some tests 
that take advantage of RDFa and SPARQL to ensure that triples for @role 
are produced as we expect.

I am volunteering to create the (very simple) tests to exercise the 
module implementations in a validation context.  I will even see if I 
can shoehorn such tests into the W3C framework people tend to use for 
W3C test cases.... although I can't see that mapping to this terribly well.

If people have a better idea, I am all ears.  If people think we don't 
need tests at all, I guess I can support that too - I am not really 
looking for more work.

-- 
Shane P. McCarron                          Phone: +1 763 786-8160 x120
Managing Director                            Fax: +1 763 786-8180
ApTest Minnesota                            Inet: shane@aptest.com

Received on Saturday, 18 August 2007 15:28:42 UTC