DRAFT Minutes QAWG TelCon 2004-Feb-02

DRAFT MINUTES

QA Working Group Teleconference
Monday, 02-February-2004
--
Scribe: Andrew

Attendees:
(PC) Patrick Curran (Sun Microsystems)
(KD) Karl Dubost (W3C, WG co-chair)
(DH) Dominique Hazaël-Massieux (W3C)
(LH) Lofton Henderson (CGMO - WG co-chair)
(DM) David Marston (IBM - guest)
(LR) Lynne Rosenthal (NIST - IG co-chair)
(MS) Mark Skall (NIST)
(AT) Andrew Thackrah (Open Group)

Regrets:
(dd) Dimitris Dimitriadis (Ontologicon)
(SM) Sandra Martinez (NIST)
(VV) Vanitha Venkatraman (Sun Microsystems)

Absent:
(MC) Martin Chamberlain (Microsoft)

Summary of New Action Items
[Format:  AI-YYYYMMDD-N   Who    What    DEADLINE! ]

Agenda: [@@@ TODO]
Previous Telcon Minutes: [@@@ TODO]

Minutes:
1.) Roll call
 see above

2.) routine business
        - Future telecons: see agenda ref [0]
        - any other?

DM  - There has been no activity on proposed panel discussion for conf.
Jan 31st deadline has elapsed. Don't know what day it would be

LH - What days would we do QA interest workshop? Karl?
KD - No fixed dates yet


3.) Test Questionnaire (DH)
        - approval of final draft [1]

DH - any further comments? - before I send to chair (+team contacts)

[no comments]

4.) In-reach reports (KD)
        - Karl to summarize, solicit reports/status

KF - WCAG: They have replied positively to my request and are implementing QA framework.
They will soon write a test suite - so it's going well.

OWL: as you know we had a WG review + a personal review from one member - dealing with this
re. I18N - have contacted them a few times - but had no answers
DOM: said 'no'

LH - I have an assisngment to do something about SVG. I think we have 5 people who have
in-reach assignments - do we still have the complete list? - if so can we make it
available to QAWG?
Re. SVG:  they've not done much re. OpsGL. SVG has a good record of tracking spec. with test suite.
They probably wont have time to spare on OpsGL due to current heavy workload.
As a WG member I will fill out QA process doc template myself - should identify gaps.

There are discussions re. further partitioning of the SVG spec using DOV - [@@@ see QAWG ref.]
I sent a comment to SVG list - got no response (they are focussed on closing tech issues).
This is a real challenge since SVG is already partitioned so I want to raise this topic
again at some point.

David, one of your comments about SVG DOV was re. prioritisation/sequencing of DOV.
Today I will publish an SVG deliverable. I believe, having done this, that it's a
very difficult problem for users of SpecGL to address the checkpoints of DOV.
ET could really use some material to explain this, maybe using your priority scheme as an example

DM - yes, or a series a questions in same sequence.


5.) TA list for SpecGL (continuing at GL8?) (MS)

MS - First new comment is 8.5
LH - What about 8.2 from a previous issue?
MS - I checked this - it's no longer a problem for me.
On 8.5: I'm not sure we define 'conformance designations' well enough to be able to test this

LR - What is the purpose of this checkpoint?
MS - To make sure that the term "conformance designation" is defined. If extensions are allowed
- to define the policy

LR - I disagree (not specifically about extensions). I think it says that if you use various
definitions about conformance - 'well formed', 'validated', 'using levels' etc - then
you should identify & define the vocabulary
Consider 'strict conformance' - this could be a designation.

MS - So we exclude 'strict conformance' from then definition requirements?
PC - Agree with LR - they have to define all terms - this is what it means
MS - Do we all agree that 'conformance designation' is too vague - so what do we need to say?
LR - Are we trying to say - "if you define conformance using special vocabulary - you must
describe the terms"?

MS - I think it is more specific: it's about conformance *groupings* - sets, levels, AAA etc
DM -  [i.e] variability
LR - But I would include terms like 'well formed', 'validating'
MS - No, it would not cover these.
LR - So would conformance policy cover these?
MS - Let's go ofline - it's very confusing
LR - Ok, do others agree that this is about groupings?
PC - Yes, general term definitions should be covered elsewhere
DM - But 8.5 may be the only checkpoint where definition of these terms is covered
MS - There is also the general checkpoint on definition of terms [8.2] - this should be wide enough?
AT - So is 8.5 a specialisation of 8.2
LR - Looks like it
MS - 8.5 is asking for more than a simple definition
PC - I think the difference [between 8.2 and 8.5] will be too subtle for users of SpecGL
LR - I will look offline for the old rationale about why we separated these requirements

MS - Moving on to 9.1: any comments?
DM - yes - caution over use of the term 'level' as opposed to 'degree'
LR - or 'designation' even
DM - I think 'degree'
LH - Unless they define degrees as their designation [!]....harking back to 8.5
MS - Ok, so 'level' becomes 'degree'.

MS - 9.2. Old rationale: disclaimer says not 100% correct. currently too weak.
Do we need to tighten it up? [For non-syntax checkers] meeting all requirements is no guarantee that
 conformance means 100% conformance to spec.

LH - It's the same with testing - 100% PASS rate does not imply 100% conformance. If we say that
for specGL conformance disclamier - whats the point of saying it?

LR - We could give an example in ExTech. It was once more specific - but we decided to move it out
to ExTech and keep the checkpoint less specific.

LH - So what is the nature of this disclaimer?
MS - Yes, what is someone supposed to do with it? copy it? if so should be exact and not a
meta-disclaimer. do we need this checkpoint - maybe not?

DM - That's the nub of the question I think
MS - Do we have a statement elsewhere - in the test suite requirements?
PC - This is a warning to the implementer to avoid making 100% claims.
LH - So getting a 100% score on the checklist means you do not conform to this doc? That can't be
right!

LR - Note that this is a priority 3 checkpoint - maybe should we wipe the whole GL?
MS - No one knows how to use it (9.2)?
LH - I'd like more time to consider before wiping the whole GL. I have sympathy with losing 9.2
though.

LR - Seems like 9.2 is lined up for removal
MS - We could check to make sure a spec conforms to SpecGL? but why have a checkpoint that
can never be checked (e.g. checking an implementation)?

LR - It still gives users advice on what to look for when making a conformance claim
 For example WAI/WCAG requires self-test honesty - and so it gives them a potted claim text, even
 though it isn't explicitly checked.

LR - Any claim is testable though
MS - But we don't know when someone is going to make a claim...lets move on

DM - 9.5 wording change - I will send email

MS - 10.1 same question about something being inline or referenced - I think we've discussed this before.
LH - Yes, the solution was to link the word 'provide' to explain that

DM - Problem here: 'cover all requirements' part turns role of spec assessor into a coverage expert.
We may be asking too much of them.

LR - They can list any [number of] test assertions - personally I want to remove and redo all of
section 10.

PC - I will reopen this by email as I'm confused.
MS - Should not re-open without specific reason
LH - it has a history - I want it re-opened too.
LR - I am adding an independent comment on the CR (not relating to historical issue)
LH - ok, but we still have to consider it's history

PC - We still haven't resolved my concern about duplication...
LH - This will be one of the big issues for CR discussion

Adjourned

Received on Tuesday, 3 February 2004 12:17:23 UTC