About TestRequirement

Hello all,

I'd like to share my concerns about TestRequirement with the group.

Initial disclaimer: Is possible that the content of this message make sense only for its author. If you read the message and you think that I am going crazy, please ignore the whole message. ;o)

I find the use of the TestRequirement term in the document confused, because I think it's not clear the difference between a requirement and a test.

IMO a requirement is a documented need of what a particular product should be or do (what you expect from the product) and test are process used to ensure products are designed and produced to meet those requirements (the way you test whether you have what you expected or not).

Based on this opinion, requirements could be:

- Guidelines
- Checkpoints
- Principles
- Success criteria
- etc.

But they shouldn't be test cases. You have requirements and then you can have one or several tests per requirement. Those tests are part of the methodology, not requirements.

If we agree on the previous understanding of both terms, I think there are several parts of the draft where the requirement and test concepts are used in a confused way:

- 1.1 Structure of EARL Results

"Test criteria
    What are we evaluating the test subject against? This could be a specification, a set of guidelines, a test from a test suite, or some other test case."

An alternative proposal is:

Test criteria
    What are we evaluating the test subject against? This could be a specification, a set of guidelines a group of checkpoints or some other success criteria.

- 2.4 Test Requirement

"A Test Requirement is a test - usually one that can be passed or failed. This includes things such as validation requirements, code test cases, checkpoints from guidelines such as Web Content Accessibility Guidelines 1.0 [WCAG10], or others. These should be identified with a URI"

An alternative proposal is:

A Test Requirement is a documented need of what a particular product should be or do - usually one that can be meet or not. This includes things such as validation requirements, checkpoints, principles, success criteria, guidelines such as Web Content Accessibility Guidelines 1.0 [WCAG10], or others. These should be identified with a URI.

The same happens at:

"A Test Requirement may be a single test, or may be a part of a larger compound test suite. These relations may be described using Dublin Core's dct:hasPart or dct:isPartOf properties..."

And an alternative proposal is:

"A Test Requirement may be a single requirement, or may be a part of a larger compound requirements suite. These relations may be described using Dublin Core's dct:hasPart or dct:isPartOf properties..."

For the same reasons I find the following examples confused too:

- Examples 1 and 2

Context
    Mary Thompson claims on the 17th December
Test Subject
    A Web page at http://www.example.org/mypage
Test Result
    Passed
Test performed
    Checkpoint 1.1 of the Web Content Accessibility Guidelines 1.0

Make more sense for me as:

Context
    Mary Thompson claims on the 17th December
Test Subject
    A Web page at http://www.example.org/mypage
Test Result
    Passed
--> Tested Criteria <--
    Checkpoint 1.1 of the Web Content Accessibility Guidelines 1.0

- Example 3

<earl:Assertion rdf:ID="#assertion">
  <earl:assertedBy rdf:resource="#assertor"/>
  <earl:subject rdf:resource="#subject"/>
  <earl:requirement rdf:resource="#testcase"/>   
  <earl:result rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#pass"/> 
  <earl:methodology rdf:resource="http://example.com/some/method#used"/>
</earl:Assertion>

Looks better for me as:

<earl:Assertion rdf:ID="#assertion">
  <earl:assertedBy rdf:resource="#assertor"/>
  <earl:subject rdf:resource="#subject"/>
-->  <earl:requirement rdf:resource="#requirement"/>  <--
  <earl:result rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#pass"/> 
  <earl:methodology rdf:resource="http://example.com/some/method#used"/>
</earl:Assertion>

- Example 8

<earl:TestRequirement rdf:ID="html">
  <dc:title xml:lang="en">HTML Test Suite</dc:title>
  <dc:description xml:lang="en">Tests specifically for the Hyper Text Markup Language</dc:title>
  <dct:isPartOf rdf:resource="http://example.org/tests/#all"/>
  <dct:hasPart rdf:resource="http://example.org/tests/#282"/>
  <dc:location>http://example.org/tests/html/</dc:location>
</earl:TestRequirement>

Looks better for me as:

<earl:TestRequirement rdf:ID="html">
  <dc:title xml:lang="en">xthml Validity</dc:title>
  <dc:description xml:lang="en">xhtml Conformance to W3C Recommendations</dc:title>
  <dct:isPartOf rdf:resource="http://example.org/tests/#all"/>
  <dct:hasPart rdf:resource="http://example.org/tests/#well-formedness"/>
  <dct:hasPart rdf:resource="http://example.org/tests/#grammar-validation"/>
  <dc:location>http://example.org/tests/xhtml/</dc:location>
</earl:TestRequirement>


Does something of this make sense for anybody else than me?

Regards,

CI.

 
--------------------------------------

Carlos Iglesias

Web Accessibility Unit - CTIC Foundation
Science and Technology Park of Gijón
33203 - Gijón, Asturias, Spain 

phone: +34 984291212
fax: +34 984390612
email: carlos.iglesias@fundacionctic.org
URL: http://www.fundacionctic.org

Received on Tuesday, 17 January 2006 18:14:29 UTC