W3C home > Mailing lists > Public > www-qa-wg@w3.org > December 2004

Test FAQ

From: Lynne Rosenthal <lynne.rosenthal@nist.gov>
Date: Wed, 22 Dec 2004 13:47:17 -0500
Message-Id: <5.1.0.14.2.20041222131153.00aa57f8@mailserver.nist.gov>
To: www-qa-wg@w3.org

Comments on Test FAQ

In general, the document is addressing an excellent set of items 
(questions), is at the right level and tone.

A suggestion is to number the questions so we can more easily identify 
which one we are addressing.

1.  What kinds of testing,
a)  5th paragraph: "Somewhat less attention seems to be focused on 
conformance test development".
We should state this more positive, especially since we are proponents of 
conformance.  Additionally, I disagree, there is more and more focus on 
conformance within W3C and in other organizations.  Suggest 
rewording:  Over the last several years, more attention has been focused on 
conformance test development and more groups have been developing 
conformance tests.

b) last para.
s/these kinds of testing/conformance and interoperability testing/

2. When should test develop start
a) 1st and 3rd paragraph:  It may be confusing that para 1 says plan early 
and para 3 says, its best to wait to develop tests.  I understand that it 
is addressing different aspects, but it took me a second to realize that - 
the words 'early' and 'wait' caught my attention, and I had to reread to 
understand the difference.  Is there a better way to describe this?  This 
is not an objection to what is there, but a question to consider.

b) 2nd para, 1st sentence
s/define/identify/

3. How should I package and publish
a) para 3. how to make a conformance claim
What is meant here.  SpecGL discusses how to make a conformance 
claim.  Does this mean the same thing?  If so, I object to it being here - 
we shouldn't encourage test developers to do this.

b) para 3 last sentence.  A complete test suite will contain some or all of 
the following.
Given this statement and that 'tests' are in the list, then can I conclude 
that a test suite can contain only 'tests' ? Or that 'tests' are NOT part 
of a test suite.  Perhaps, since tests are a given, reword as: "A complete 
test suites will contain tests and some or all of the following"

4. What should the test documentation cover
Again, how to make a conformance claim item should be deleted or clarified

5. Should I automate test execution
What is missing here is something we found out - if you make it easy, 
people will use the tests.  Suggest we add something along this line.

regards
Lynne
Received on Wednesday, 22 December 2004 18:52:21 GMT

This archive was generated by hypermail 2.2.0 + w3c-0.30 : Thursday, 9 June 2005 12:13:19 GMT