- From: Lofton Henderson <lofton@rockynet.com>
- Date: Wed, 18 Feb 2004 18:06:02 -0700
- To: olivier Thereaux <ot@w3.org>
- Cc: www-qa-wg@w3.org
At 09:55 AM 2/17/04 +0900, olivier Thereaux wrote: >Hello QAWG, Hi Olivier, >As you know, the WCAG working group is taking QA seriously and have >been trying to follow our guidelines as closely as possible. I usually >try to answer question from Jenae Andershonis (QA contact within WCAG >WG), but she recently sent me a batch for which I could use your >insight, These are only my own interpretations -- no QAWG standing (would QAWG consensus answers be a good idea?). >and I thought you'd be interested in the feedback they >provide, too... > >Specl GL: >>I have a question about Checkpoint 7.1. Use conformance key words. The >>latest structure for WCAG 2.0 is Principle | Guideline | Level (1, 2 >>or 3) Success Criteria. I'm looking at: [0] http://www.w3.org/TR/WCAG20/#perceivable I see some Success Criteria. But I don't see Levels, Principles, etc. So I assume I'm not looking at "the latest structure". >>My question is at what level should the RFC 2119 keywords be applied? >>Should the keywords be used in the statement for Principle, Guideline >>or Success Criteria? Or at all three? My guess is that it should be at >>the Success Criteria level. > >So do I. What do you think? However, looking at that /TR/ version ... yes, the RFC keywords would potentially be applicable within the Success Criteria. EXCEPT ... WCAG has chosen a different style to express its basic testable statements. They have chosen a conventional "test assertion" (TA) style. RFC keywords would not be applicable unless they changed to a (SpecGL-like) "conformance requirements" style. E.g., "1. non-text content that can be expressed in words has a text-equivalent explicitly associated with it." would become, "1. non-text content that can be expressed in words MUST have a text-equivalent explicitly associated with it". Note that CP7.1 allows the current WCAG style (TAs), as long as the style is explained and the reason for using it instead of RFC keywords is explained. (E.g., "less fierce" or "don't shout" or ...). Aside. As a result of the CR trial implementation and review period for SpecGL and OpsGL, there are newly re-opened issues, challenging the RFC-keywords style should be so strongly preferred in all cases. My personal recommendation (for which QAWG may lynch me!) ... don't change this aspect of WCAG20, at least not until QAWG has had the chance to sort this issue again. >Test GL (based on TR version) Btw, we hope to publish (/TR/) a new version of TestGL next week, and I think it will be much changed from the current /TR/ version that Jenea is using (dated May 2003) . The new version's most recent editor-draft ancestor is: [1] http://www.w3.org/QA/WG/2003/10/TestGL-20031020.html >>Question #1 >>Checkpoint 4.3. Automation of testing is encouraged. >> >>We weren't sure if this meant that the test cases should be >>automatable, (i.e. doesn't involve a human opinion). Or if it means >>that we should engage with vendors of tools like A-Prompt and Bobby, >>or does it mean something even different than these two >>interpretations. > >Not sure what Jenae means by "engaging" here, but my interpretation of >the GL is that the tests should be automatable when possible. (which, >for WCAG, is not always/necessarily/easily possible) I'm not sure I understand the question exactly. So I'll say some things and see if anything hits a chord. I see two possible applications of automation: 1.) the test case itself is automated (doesn't involve human intervention); 2.) the test *framework* is automated. I think that the /TR/ version of TestGL CP4.3 is about automation of the framework, #2. I.e., even if the individual test case involves human judgement and intervention (WCAG rule assessment, VBWG subjective test, OpsGL CP conformance), there can be an automated framework (e.g., harness) that sequences the tests, presents the tester with a nice form-driven questionnaire, assembles the results, and reports "the final score". We were talking about this very thing, this morning, in QAWG telecon, considering "what are Test Materials for GL documents like OpsGL?" Aside. The W3C Markup Validator is a pure example of #1. As I understand Bobby, it also is a #1-ish of test tool (i.e., it's a WCAG checker, right? Reads the document and tries to analyze it from perspective of WCAG rules?). >>Question #2 >>Checkpoint 6.1. Organize conformance testing activities. >>Would the definition of vendor be like Watchfire (Bobby tool)? > >I think this is an interesting one... AFAIK vendors here mean >implementors of the specification, which does not necessarily mean >anything for Guideline specifications... A wording warning for the QAF, >I suppose. Interesting question. Let's see, the Class of Product (CoP) of WCAG -- i.e., the object or target of its conformance requirements -- is Web documents. Therefore, an "implementation" would be an instance of a Web document? And a "vendor of an implementation" would be an author/owner of Web document instance(s)? I think that's a sound chain of inference. That said, might one generalize "author" to authoring tool? Yes, I know that there is an ATAG. I'm not thinking about all of the aspects of an authoring tool that ATAG addresses. I'm thinking of a single aspect -- the WCAG-compliance of the documents it produces. Yes ... we should look at QAF and make sure that the "content" CoP (which is WCAG's CoP) fits into all of our GL & CP about testing, test materials, etc. By the way, these test-activity checkpoints of that 5/2003 TestGL draft have be moved into OpsGL: [2] http://www.w3.org/TR/2003/CR-qaframe-ops-20030922/guidelines-chapter#Ck-publish-test-results >No "vendor" entry in the QA glossary it seems: >http://www.w3.org/2003/glossary/keyword/qa-glossary.rdf/?keywords=vendor *Lots* of stuff missing. WebOnt and JC have made some serious comments about that. >>Question #3 >>Checkpoint 6.2. Encourage Vendors to publish test results. >>Is the preferred "Special place" within the W3C site or a link to the >>vendor's site? >>And should we create something like the 508 VPAT for the vendor to >>complete? > >IIRC, for test result publication, W3C is preferred, though not >strongly. Correct? Yes. See [2]. 508 VPAT? Not sure what that is. Sounds like a standardized results reporting form? This brings to mind SpecGL's Implementation Conformance Statement (ICS), remembering that "implementation" here is "Web page instance": [3] http://www.w3.org/TR/2003/CR-qaframe-spec-20031110/guidelines-chapter#Ck-require-ICS >>Question #4 >> >>Our proposed charter [2] states that we would commit to attaining QA >>level three. Now that the conformance structure has been changed to A, >>AA and AAA, what would level three equate to? And do we need to list >>out of conformance goals for all three of the QA specs. >> >>[2] http://www.w3.org/2003/08/wcag-charter.html > >I think this has been simplified and corrected in Ops, correct? Yes. In the LC version of OpsGL, that table bore no relationship to the requirements of the rest of the QAF family, and it was heavily criticized. Therefore we revised it to: tell us how you intend to conform to OpsGL, to SpecGL, and to TestGL. >http://www.w3.org/TR/2003/CR-qaframe-ops-20030922/guidelines-chapter#Gd-address-QA-in-charter >I do not, however, recall what level three would be equivalent to (A or >AA I guess), could anyone tell me? Although the current GL1 checkpoints are simpler and clearer, it isn't a simple mapping between them and the old LC version of OpsGL. Old Level 3 required, for example, a set of Test Assertions. That is P3 in SpecGL (hence only needed for AAA SpecGL conformance), and P1/P2 in TestGL (so needed for A/AA TestGL conformance). >Hope this list of question provides you with interesting feedback, and >that you can help me give good answers to this very motivated group. Again ... would it be a good idea to generate a WG consensus response? -Lofton.
Received on Wednesday, 18 February 2004 20:03:10 UTC