Various Test/Spec questions from WCAG

Hello QAWG,

As you know, the WCAG working group is taking QA seriously and have  
been trying to follow our guidelines as closely as possible. I usually  
try to answer question from Jenae Andershonis (QA contact within WCAG  
WG), but she recently sent me a batch for which I could use your  
insight, and I thought you'd be interested in the feedback they  
provide, too...

Specl GL:
> I have a question about Checkpoint 7.1. Use conformance key words. The  
> latest structure for WCAG 2.0 is Principle | Guideline | Level (1, 2  
> or 3) Success Criteria.
> My question is at what level should the RFC 2119 keywords be applied?  
> Should the keywords be used in the statement for Principle, Guideline  
> or Success Criteria? Or at all three? My guess is that it should be at  
> the Success Criteria level.

So do I. What do you think?

Test GL (based on TR version)
> Question #1
> Checkpoint 4.3. Automation of testing is encouraged.
>
> We weren’t sure if this meant that the test cases should be  
> automatable, (i.e. doesn’t involve a human opinion). Or if it means  
> that we should engage with vendors of tools like A-Prompt and Bobby,  
> or does it mean something even different than these two  
> interpretations.

Not sure what Jenae means by "engaging" here, but my interpretation of  
the GL is that the tests should be automatable when possible. (which,  
for WCAG, is not always/necessarily/easily possible)


> Question #2
> Checkpoint 6.1. Organize conformance testing activities.
> Would the definition of vendor be like Watchfire (Bobby tool)?

I think this is an interesting one... AFAIK vendors here mean  
implementors of the specification, which does not necessarily mean  
anything for Guideline specifications... A wording warning for the QAF,  
I suppose.

No "vendor" entry in the QA glossary it seems:
http://www.w3.org/2003/glossary/keyword/qa-glossary.rdf/?keywords=vendor

> Question #3
> Checkpoint 6.2. Encourage Vendors to publish test results.
> Is the preferred “Special place” within the W3C site or a link to the  
> vendor’s site?
> And should we create something like the 508 VPAT for the vendor to  
> complete?

IIRC, for test result publication, W3C is preferred, though not  
strongly. Correct?

> Question #4
>
> Our proposed charter [2] states that we would commit to attaining QA  
> level three. Now that the conformance structure has been changed to A,  
> AA and AAA, what would level three equate to? And do we need to list  
> out of conformance goals for all three of the QA specs.
>
> [2] http://www.w3.org/2003/08/wcag-charter.html

I think this has been simplified and corrected in Ops, correct?
http://www.w3.org/TR/2003/CR-qaframe-ops-20030922/guidelines- 
chapter#Gd-address-QA-in-charter
I do not, however, recall what level three would be equivalent to (A or  
AA I guess), could anyone tell me?


Hope this list of question provides you with interesting feedback, and  
that you can help me give good answers to this very motivated group.
-- 
olivier