[techs] Addendum to minutes

Just moments after I sent the minute I received more detailed notes from the
discussion on QA Test Guidelines. Here they are:




We also began our discussion of the QA Framework: Test Guidelines [1]. The
spec is in the draft state. This spec will have an 
accompanying document for Examples and Techniqueswhich has not been posted
yet.



Today we discussed Checkpoints 1.1 through 4.1. January 7, we will resume 
our discussion



Checkpoint 1.1. Define the test suite scope [Priority 1]

The QAPD document [2] also suggests defining our test suite 
scope. Action Item for Jenae: Draft our requirements for techs scope, 
refer to proposed charter.



Checkpoint 1.2. Identify the specification(s) to be tested. [Priority 1]

WCAG 2.0

Action Item for Jenae: Double check Olivier is it this simple that we just 
need to state that the spec we are testing is WCAG 2.0



Checkpoint 1.3. Analyze the structure of the specification, partition it 
as appropriate, and determine and document the testing approach to be used 
for the test suite as a whole and for each partition. [Priority 1]



Testing against success criteria? Testing approach? Approaches may be 
different for different checkpoints.

Are we testing the guideline or the techniques? Action Item for Jenae: 
create draft



Checkpoint 2.1. Identify and list assertions within the specification. 
[Priority 1]

assertions = success criteria? Define assertions? Can you create a 
technique for a guideline checkpoint?



Wendy summarized this well:

Apply to normative information WCAG 2.0.

Two levels of test

1.         testing WCAG 2.0

2.         testing the techniques





Checkpoint 2.2. Metadata must be associated with test assertions, enabling 
test developers, the test-management system, the test-execution framework, 
and the results-reporting process to make useful distinctions between 
groups of tests. [Priority 2]



Partial addressed this with Chriswork. We use XML. Why have the metadata 
and how would it be used?





Checkpoint 3.1. The Working Group must provide a test management system 
[Priority 1]

Michael and Chriswork still need to define what this means in terms of QA. 
Do we want to generalize the work of M and C? What is our test management 
system?



Checkpoint 3.2. The test management system must associate tests with 
metadata. [Priority 1]

Checkpoint 3.3. Test management system must allow filtering based on 
metadata. [Priority 2]



DTD is structured to allow that.



Checkpoint 3.4. Test management system must support results. [Priority 2]



Checkpoint 4.1. Provide a test framework [Priority 1]





Action Item for Jenae: Follow up with Olivier with our group questions. 
Please note that Olivier is on vacation now, and Ill be on vacation when 
he returns. I may not have answers for the group until the second week of 
January.


[1] 
<http://www.w3.org/TR/2003/WD-qaframe-test-20030516/>http://www.w3.org/TR/20
03/WD-qaframe-test-20030516/

[2] 
<http://lists.w3.org/Archives/Public/w3c-wai-gl/2003OctDec/0568.html>http://
lists.w3.org/Archives/Public/w3c-wai-gl/2003OctDec/0568.html