Final Minutes QAWG Telcon 2004-Feb-02

FINAL MINUTES

QA Working Group Teleconference
Monday, 02-February-2004
--
Scribe: Andrew

Attendees:
(PC) Patrick Curran (Sun Microsystems)
(KD) Karl Dubost (W3C, WG co-chair)
(DH) Dominique Hazaël-Massieux (W3C)
(LH) Lofton Henderson (CGMO - WG co-chair)
(DM) David Marston (IBM - guest)
(LR) Lynne Rosenthal (NIST - IG co-chair)
(MS) Mark Skall (NIST)
(AT) Andrew Thackrah (Open Group)

Regrets:
(dd) Dimitris Dimitriadis (Ontologicon)
(SM) Sandra Martinez (NIST)
(VV) Vanitha Venkatraman (Sun Microsystems)

Absent:
(MC) Martin Chamberlain (Microsoft)

Summary of New Action Items
[None]

Agenda: http://lists.w3.org/Archives/Public/www-qa-wg/2004Feb/0002.html
Previous Telcon Minutes: 
http://lists.w3.org/Archives/Public/www-qa-wg/2004Feb/0008.html

Minutes:
1.) Roll call
 see above
 
2.) routine business
        - Future telecons: see agenda ref
        - any other?

DM  - There has been no activity on proposed panel discussion for conf.
Jan 31st deadline has elapsed. Don't know what day it would be

LH - What days would we do QA interest workshop? Karl?
KD - No fixed dates yet


3.) Test Questionnaire (DH)
        - approval of final draft

DH - any further comments? - before I send to chair (+team contacts)

[no comments]

4.) In-reach reports (KD)
        - Karl to summarize, solicit reports/status

KF - WCAG: They have replied positively to my request and are 
implementing QA framework.
They will soon write a test suite - so it's going well.

OWL: as you know we had a WG review + a personal review from one member 
- dealing with this
re. I18N - have contacted them a few times - but had no answers
DOM: said 'no'

LH - I have an assignment to do something about SVG. I think we have 5 
people who have
in-reach assignments - do we still have the complete list? - if so can 
we make it
available to QAWG?
Re. SVG:  they've not done much re. OpsGL. SVG has a good record of 
tracking spec. with test suite.
They probably won't have time to spare on OpsGL due to current heavy 
workload.
As a WG member I will fill out QA process doc template myself - that 
should identify any gaps.

There are discussions [0] re. further partitioning of the SVG spec using 
DOV
I sent a comment to SVG list - got no response (they are focussed on 
closing tech issues).
This is a real challenge since SVG is already partitioned so I want to 
raise this topic
again at some point.
 
David, one of your comments about SVG DOV was re. 
prioritisation/sequencing of DOV [1].
Today I will publish a SpecGL-analysis deliverable. I believe, having 
done this, that it's a
very difficult problem for users of SpecGL to address the checkpoints of 
DOV.
ET could really use some material to explain this, maybe using your 
priority scheme as an example

DM - yes, or a series a questions in the same sequence.


5.) TA list for SpecGL (continuing at GL8?) (MS)
        
MS - First new comment is 8.5
LH - What about 8.2 from a previous issue?
MS - I checked this - it's no longer a problem for me.
On 8.5: I'm not sure we define 'conformance designations' well enough to 
be able to test this

LR - What is the purpose of this checkpoint?
MS - To make sure that the term "conformance designation" is defined. If 
extensions are allowed
- to define the policy

LR - I disagree (not specifically about extensions). I think it says 
that if you use various
definitions about conformance - 'well formed', 'validated', 'using 
levels' etc - then
you should identify & define the vocabulary
Consider 'strict conformance' - this could be a designation.

MS - So we exclude 'strict conformance' from the definition requirements?
PC - Agree with LR - they have to define all terms - this is what it means
MS - Do we all agree that 'conformance designation' is too vague - so 
what do we need to say?
LR - Are we trying to say - "if you define conformance using special 
vocabulary - you must
describe the terms"?

MS - I think it is more specific: it's about conformance *groupings* - 
sets, levels, AAA etc
DM -  [i.e] variability
LR - But I would include terms like 'well formed', 'validating'
MS - No, it would not cover these.
LR - So would conformance policy cover these?
MS - Let's go ofline - it's very confusing
LR - Ok, do others agree that this is about groupings?
PC - Yes, general term definitions should be covered elsewhere
DM - But 8.5 may be the only checkpoint where definition of these terms 
is covered
MS - There is also the general checkpoint on definition of terms [8.2] - 
this should be wide enough?
AT - So is 8.5 a specialisation of 8.2?
LR - Looks like it
MS - 8.5 is asking for more than a simple definition
PC - I think the difference [between 8.2 and 8.5] will be too subtle for 
users of SpecGL
LR - I will look offline for the old rationale about why we separated 
these requirements

MS - Moving on to 9.1: any comments?
DM - yes - caution over use of the term 'level' as opposed to 'degree'
LR - or 'designation' even
DM - I think 'degree'
LH - Unless they define degrees as their designation [!]....harking back 
to 8.5
MS - Ok, so 'level' becomes 'degree'.

MS - 9.2. Old rationale: disclaimer says not 100% correct. currently too 
weak.
Do we need to tighten it up? [For non-syntax checkers] meeting all 
requirements is no guarantee that
 conformance means 100% conformance to spec.

LH - It's the same with testing - 100% PASS rate does not imply 100% 
conformance. If we say that
for SpecGL conformance disclaimer - what's the point of saying it?

LR - We could give an example in ExTech. It was once more specific - but 
we decided to move it out
to ExTech and keep the checkpoint less specific.

LH - So what is the nature of this disclaimer?
MS - Yes, what is someone supposed to do with it? copy it? if so should 
be exact and not a
meta-disclaimer. do we need this checkpoint - maybe not?

DM - That's the nub of the question I think
MS - Do we have a statement elsewhere - in the test suite requirements?
PC - This is a warning to the implementer to avoid making 100% claims.
LH - So getting a 100% score on the checklist means you do not conform 
to this doc? That can't be
right!

LR - Note that this is a priority 3 checkpoint - maybe should we remove 
the whole GL?
MS - No one knows how to use it (9.2)?
LH - I'd like more time to consider before wiping the whole GL. I have 
sympathy with losing 9.2
though.

LR - Seems like 9.2 is lined up for removal
MS - We could check to make sure a spec conforms to SpecGL? but why have 
a checkpoint that
can never be checked (e.g. checking an implementation)?

LR - It still gives users advice on what to look for when making a 
conformance claim
 For example WAI/WCAG requires self-test honesty - and so it gives them 
a potted claim text, even
 though it isn't explicitly checked.

LR - Any claim is testable though
MS - But we don't know when someone is going to make a claim...lets move on

DM - 9.5 wording change - I will send email

MS - 10.1 same question about something being inline or referenced - I 
think we've discussed this before.
LH - Yes, the solution was to link the word 'provide' to explain that

DM - Problem here: 'cover all requirements' part turns role of spec 
assessor into a coverage expert.
We may be asking too much of them.

LR - They can list any [number of] test assertions - personally I want 
to remove and redo all of
section 10.

PC - I will reopen this by email as I'm confused.
MS - Should not re-open without specific reason
LH - it has a history - I want it re-opened too.
LR - I am adding an independent comment on the CR (not relating to 
historical issue)
LH - ok, but we still have to consider it's history

PC - We still haven't resolved my concern about duplication...
LH - This will be one of the big issues for CR discussion

Adjourned

[0] http://lists.w3.org/Archives/Public/www-qa-wg/2004Jan/0045.html
[1] http://lists.w3.org/Archives/Public/www-qa-wg/2004Jan/0054.html

Received on Monday, 9 February 2004 10:29:37 UTC