Re: Questions and proposals related to strawman schema from F2F

On Thu, 26 Sep 2002, Nick Gibbins wrote:

>>> The line between a testcase and a criterion is also a bit blurry -
>>> are WCAG checkpoints considered to be testcases or criteria?
WAC:
>> In WCAG 2.0 we have 3 levels:
>> 1. Guidelines
>> 2. Checkpoints
>> 3. Success criteria
>
>> a success criterion is the child of one checkpoint and the
>> great-grandchild of one guideline.  a checkpoint is the child of one
>> guideline and at least one child (success criterion).  a guideline
>> has siblings (other guidelines) and more than one child
>> (checkpoint).  the root is the document "WCAG 2.0"
>
>> Although, people won't be claiming conformance to Guidelines, they
>> will only be testing against checkpoints and success criteria.
>> Thus, I think your proposed schema would work for WCAG 2.0
>> conformance testing. (I tried it out below...really quick first
>> pass...)

I don't think this is true. Some people will claim conformance to Section 508
for example. Then to test for conformance to WCAG level-A there are some
additional checkpoints (which might be broken up into testcases).

Also, people will have different top-level criteria, but may use the same
testcases (e.g. testing for WCAG and 508 there are a number of testcases that
would be re-used, and for testing WCAG itself there are some testcases that
are useful for more than one checkpoint). So I think it is important to
define a criterion as being satisfied by a number of testcases rather than
describing a particular test-case as belonging to a criterion.

The ability to do this is going to be important if we want to merge results
from multiple tools testing a single object - for example a couple of
automatic checks and some manual ones combined to generate a claim of WCAG
double-A conformance.

(This is related to my discussion of an "heuristic" testMode elsewhere on the
list).

cheers

Chaals

>> [...] I didn't need to use Criterion, CriterionGroup,
>> containsCriterion, or tests.  I really like the idea of tests,
>> tho...
>
>Yes, I think that I misunderstood the requirements here and
>overgenerated a bit. I was trying to draw a distinction between the
>actual testcases (eg. the CSS file that shows whether or a user agent
>renders a certain feature correctly) and the things that the testcases
>test (eg. the requirement that a certain CSS feature be correctly
>rendered for a user agent to be described as compliant). This would
>allow there to be more than one testcase which tested for compliance
>to a particular requirement (criterion).
>
>Having said that, while it is fairly easy to draw the distinction
>between testcases and criteria for user agent tests (such as Ian's CSS
>test suite), I can't think of a WCAG-like usecase that needs the
>distinction, or indeed in which it is possible to make the
>distinction.
>
>In summary, I agree that we don't need to represent the criteria which
>testcases test - as Nick put it, this is too close to a test
>definition language.

Received on Friday, 27 September 2002 10:45:08 UTC