Re: comments on test metadata

Hi Vangelis,

At 13:59 16/08/2006, Vlachogiannis Evangelos wrote:
<blockquote>
I was wondering about the usefulness of "Preconditions" element as it 
appears (if I am not missing something) both in TCDL and 
http://www.w3.org/TR/2005/NOTE-test-metadata-20050914/#preconditions-def 
(##any). In my opinion such an element to be more useful mostly to tools 
and web UIs should contain one or more xlink:href to prerequisite test 
case(s) (other TCDL instances). This also introduces an issue for a URI 
allocation (some conventions) of test cases so we can refer to them. 
Indeed, sometimes such an approach might not be applicable so a literal 
element would be also necessary (have a "choice").
</blockquote>

There are several aspects to this issue:
* Since the current definition of "Preconditions" in TCDL allows any 
elements and any attributes in any namespace, it is already possible to add 
as many xlink:hrefs as you like. However, I could not define a specific 
content model for "Preconditions" because I don't know if we will need it 
in the Task Force and for what purpose. With the ##any trick, any content 
model that we might define later will still validate against the current 
XML Schema.
* Can you clarify why you would link to other test cases? When I think of 
"preconditions", I think of somethink like tests (e.g. "the img element has 
an alt attribute" would be a precondtion for "the alt attribute contains a 
value that presents the same information as the image"). However, test 
cases are not tests in that sense, so I am very reluctant to use test cases 
as preconditions.
* Developing test cases is different from developing test procedures. Test 
procedures can have other test procedures as preconditions (see the alt 
example above), but I don't see how test cases can have other test cases as 
test procedures. What we can do however, is to use existing (and 
referenceable) test procedures as preconditions for the test cases, but I 
currently don't find this absolutely necessary.


<blockquote>
I was also thinking about baselines. I am confused about the relation 
between "technicalSpec" and baselines. As far I understand technicalSpec 
element includes technical specifications related to certain test and it 
probably does not include all technologies might be used in a test file. 
Then, "baseline" attribute can state that certain techspec is included also 
in baseline or not... but what is our baseline and in what context? Would 
it be useful to define explicitly in a meta file a "base" baseline per test 
suite? Then the baseline attribute (technicalSpec) of a tcdl instance could 
override that to offer a resulting explicit baseline of the test case in a 
differential manner. Otherwise maybe a "baseline" element could be 
introduced but this would cause a lot of repetition.
</blockquote>

The XML Schema cannot enforce that the technologies element contains a 
technicalSpec element for each technology used in a test sample, but the 
rest of the specification can state this requirement. (I know it currently 
does not).
If I rewrite the description of the technologies element to require that 
all technologies in the test sample are listed (the baseline attribute is 
already required), I think we sufficiently cover baselines. What we need to 
know is not: "What is the baseline for this test suite?" but "Is 
feature/technology X in test sample Y in the baseline or not?" This allows 
more flexibility than defining baselines outside the TCDL documents and 
referring to them. If for example, a test case uses XHTML (inside 
baseline), CSS (inside), and JavaScript (outside), this test case will also 
be valid in a context where the baseline is XHTML, CSS, PDF, and no 
JavaScript. But if we create a different test suite per baseline (or define 
a baseline per test suite, as you propose), we'll just end up duplicating 
dozens of test cases, just with a different baseline URL. Or have I 
misunderstood your comment?

Best regards,

Christophe

-- 
Christophe Strobbe
K.U.Leuven - Departement of Electrical Engineering - Research Group on 
Document Architectures
Kasteelpark Arenberg 10 - 3001 Leuven-Heverlee - BELGIUM
tel: +32 16 32 85 51
http://www.docarch.be/


Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm

Received on Wednesday, 16 August 2006 14:45:29 UTC