RE: comments on test metadata

Hi Christophe, all,

-----Original Message-----
From: public-wai-ert-tsdtf-request@w3.org
[mailto:public-wai-ert-tsdtf-request@w3.org] On Behalf Of Christophe
Strobbe
Sent: Wednesday, August 16, 2006 5:45 PM
To: public-wai-ert-tsdtf@w3.org
Subject: Re: comments on test metadata

Hi Vangelis,

At 13:59 16/08/2006, Vlachogiannis Evangelos wrote:
<blockquote>
I was wondering about the usefulness of "Preconditions" element as it
appears (if I am not missing something) both in TCDL and
http://www.w3.org/TR/2005/NOTE-test-metadata-20050914/#preconditions-def
(##any). In my opinion such an element to be more useful mostly to tools
and web UIs should contain one or more xlink:href to prerequisite test
case(s) (other TCDL instances). This also introduces an issue for a URI
allocation (some conventions) of test cases so we can refer to them. 
Indeed, sometimes such an approach might not be applicable so a literal
element would be also necessary (have a "choice").
</blockquote>

There are several aspects to this issue:
* Since the current definition of "Preconditions" in TCDL allows any
elements and any attributes in any namespace, it is already possible to
add as many xlink:hrefs as you like. However, I could not define a
specific content model for "Preconditions" because I don't know if we
will need it in the Task Force and for what purpose. With the ##any
trick, any content model that we might define later will still validate
against the current XML Schema.

>> Yes.. It is always a matter of compromising between syntactic freedom
and effective schema... We could alternatively, if we want to, just
include such a "rule" to our "guidance" document.

* Can you clarify why you would link to other test cases? When I think
of "preconditions", I think of somethink like tests (e.g. "the img
element has an alt attribute" would be a precondtion for "the alt
attribute contains a value that presents the same information as the
image"). However, test cases are not tests in that sense, so I am very
reluctant to use test cases as preconditions.

>> hmm.. It seems that my interpretation in wrong...  Well, I actually
remember during developing tests I had to develop more tests as I could
not state somewhere that "I assume this has been tested". So I would
prefer to have an element to include the prereq tests.

* Developing test cases is different from developing test procedures.
Test procedures can have other test procedures as preconditions (see the
alt example above), but I don't see how test cases can have other test
cases as test procedures. What we can do however, is to use existing
(and
referenceable) test procedures as preconditions for the test cases, but
I currently don't find this absolutely necessary.

>> I guess these are actually procedures. I am afraid I don't remember
exactly for what SCs I met that requirement ( I suspect at least the one
with most test cases??)...

<blockquote>
I was also thinking about baselines. I am confused about the relation
between "technicalSpec" and baselines. As far I understand technicalSpec
element includes technical specifications related to certain test and it
probably does not include all technologies might be used in a test file.

Then, "baseline" attribute can state that certain techspec is included
also in baseline or not... but what is our baseline and in what context?
Would it be useful to define explicitly in a meta file a "base" baseline
per test suite? Then the baseline attribute (technicalSpec) of a tcdl
instance could override that to offer a resulting explicit baseline of
the test case in a differential manner. Otherwise maybe a "baseline"
element could be introduced but this would cause a lot of repetition.
</blockquote>

The XML Schema cannot enforce that the technologies element contains a
technicalSpec element for each technology used in a test sample, but the
rest of the specification can state this requirement. (I know it
currently does not). If I rewrite the description of the technologies
element to require that all technologies in the test sample are listed
(the baseline attribute is already required), I think we sufficiently
cover baselines. 

>> This sounds fine to me in case technicalSpec aims to contain all
technologies used. 

What we need to know is not: "What is the baseline for this test suite?"

>> I actually meant "baseline for this test sample"

but "Is feature/technology X in test sample Y in the baseline or not?"
This allows more flexibility than defining baselines outside the TCDL
documents and referring to them. If for example, a test case uses XHTML
(inside baseline), CSS (inside), and JavaScript (outside), this test
case will also be valid in a context where the baseline is XHTML, CSS,
PDF, and no JavaScript. But if we create a different test suite per
baseline (or define a baseline per test suite, as you propose), we'll
just end up duplicating dozens of test cases, just with a different
baseline URL. Or have I misunderstood your comment?

>> I am afraid I didn't meant that, sorry, let me try once again: I was
thinking about only one test suite with one "default baseline" in a
similar way that a browser has a default styling for html and overrides
them with pages' css. But anyway, if technicalSpec contain all
technologies this is not necessary.

Best Regards,
Vangelis

---------------------------------------------------------------
Evangelos Vlachogiannis
Researcher - PhD. Candidate
Contact: http://www.syros.aegean.gr/users/evlach/contactme.php
---------------------------------------------------------------

Received on Thursday, 17 August 2006 09:39:26 UTC