- From: Karl Dubost <karl@w3.org>
- Date: Wed, 6 Mar 2002 14:06:39 -0500
- To: www-qa-wg@w3.org
Hi attendees
You have one week to review these minutes ;)
http://www.w3.org/QA/2002/03/01-f2f-minutes
The text version
------------------
[1]W3C [2]QA
[1] http://www.w3.org/
[2] http://www.w3.org/QA/
Conformance and Quality Assurance
[3]Action Items· [4]Minutes· [5]Item 3·
_________________________________________________________________
Nearby: [6]QA Homepage· [7]Latest News· [8]QA Resources· [9]QA IG·
[10]QA WG· [11]QA Calendar·
[6] http://www.w3.org/QA/
[7] http://www.w3.org/QA/#latest
[8] http://www.w3.org/QA/#resources
[9] http://www.w3.org/QA/IG/
[10] http://www.w3.org/QA/WG/
[11] http://www.w3.org/QA/Agenda/
Minutes of QA f2f meeting 1st March 2002
See the [12]Agenda.
[12] http://www.w3.org/QA/2002/02/F2F-agenda-20020301
Attendees:
* (LH) Lofton Henderson (CGMO - WG co-chair)
* (KD) [13]Karl Dubost (W3C, WG co-chair)
* (LR) Lynne Rosenthal (NIST - IG co-chair)
* (MS) Mark Skall (NIST)
* (OT) [14]Olivier Théreaux
* (DD) [15]Daniel Dardailler (W3C - IG co-chair)
* (DH) [16]Dominique Hazael-Massieux (W3C - Webmaster)
* (PF) Peter Fawcett (Real Networks)
* (KG) Kirill Gavrylyuk (Microsoft)
* (dd) Dimitris Dimitriadis (Ontologicon)
* Colas Nahaboo
* [17]Bert Bos
* [18]Wendy Chisholm
* [19]Charles McCathieNeville
* [20]Susan Lesch
* Carine Bournez
* [21]Yvon Lafon
[13] http://www.w3.org/People/karl/
[14] http://www.w3.org/People/olivier
[15] http://www.w3.org/People/danield
[16] http://www.w3.org/People/dom/
[17] http://www.w3.org/People/Bos
[18] http://www.w3.org/People/wendy
[19] http://www.w3.org/People/Charles/
[20] http://www.w3.org/People/Lesch/
[21] http://www.w3.org/People/lafon
Summary of Action Items
* A-2002-03-1-1 Karl Clarify proposal about [22]summary of QA IG
(and WG) mailing lists.
* A-2002-03-1-2 Olivier and Lofton Post revised tech plenary
slides.
* A-2002-03-1-3 Dimitris Develop proposal for formation of Test
Group (including coordination to make testing a W3C critical
effort.)
* A-2002-03-1-4 Karl Develop proposal as to how to get WGs to
effectively assign resources for testing activities
* A-2002-03-1-5 Karl and Lofton Put [23]matrix of document family
on Working Group page with links to published versions.
* A-2002-03-1-6 PF Go through mail archives and write up 4-5 top
justifications for why we don't think test materials should be
published in TR space.
* A-2002-03-1-7 LH Split issue #44 into 2 issues. The first dealing
with the 'ranking' of test materials and the second dealing with
the 'endorsement of test materials.
* A-2002-03-1-8 LH Get bridge for Thursday (March 7) Telecon.
* A-2002-03-1-9 KD Claims ownership of process document for QAWG.
* A-2002-03-1-10 KD To look into possibility of finding place for
Face to Face in Montreal.
[22] http://www.w3.org/QA/2002/02/aweekinqa
[23] http://www.w3.org/QA/WG/#docs
Meeting minutes - Morning
Roll Call
Membership Topics
Rob Lanphier is no longer a member of the Working Group; he is just an
observer. A discussion ensued concerning the need for more members.
Although it was agreed that the QA Working Groupís efforts are
becoming much more visible, the consensus was that having more members
would allow the Group to maintain our ambitious schedule and large
amount of deliverables.
Outreach and Education
Karl Dubost talked about his activity to develop an overview about
whatís going on the QA mailing list. It will be interesting for people
(journalist, web community as large, occasional readers) to have an
abstract of what has been discussed on the mailing list. The goal is
both to inform people who are not involved in our activity as well as
to encourage people to join the activity.
The original proposal was to create a pool of 3 people in charge of
writing this overview each week, alternatively. A team member of the
QA activity will review the overview. It was decided to include the
Interest Group and any relevant Working Group traffic in the
overview. A discussion ensued concerning whether weekly was too often
to issue the overview. Monthly was also proposed but that was thought
not often enough. It was finally decided to produce the overview
bi-weekly. It was decided to try this for 6 months and then
re-evaluate the utility of the overview.
* Action Item - Karl Clarify proposal about summary of QA IG (and
WG) mailing lists.
* Action Item - Olivier and Lofton Post revised tech plenary
slides.
Liaisons
The following liaison reports were given:
* CSS (Lofton) Good feedback was received on the matrix. CSS was
unaware of the QA activity.
* ATAG (Lofton) ATAG techniques document is relevant to our
examples and techniques document.
* HTML (Lofton) Lofton made presentation. They liked the matrix.
* XML Core (Lofton, Lynne) Entry in matrix needs to be updated.
NIST/OASIS test suite transferred to XML/Core. Tests will be
revised and fixed. Test suite will be published as a Working
Group deliverable.
* WAI EO (Daniel) Better way to implement WAI Guidelines by better
reviews of web sites using the NIST (Skall, Rosenthal) white
paper.
* DOM (Dimitris) Need resources allocated for test materials.
A discussion then ensued regarding how to get Working Groups to
allocate enough resources to do an effective job in testing and in
meeting the QA requirements. There were two alternatives proposed:
1. A liaison should be identified in each Working Group. This
liaison would be responsible for ensuring that the Working Group
meet all of the QA Groupís requirements (proposed by Mark);
2. The W3C team should provide liaisons to the Working Group to
ensure that the Working Groups meet all the QA requirements
(proposed by Karl);
3. Coordinate testing to make it a W3C critical effort (proposed by
Dimitris).
* Action Item - Dimitris Develop proposal for formation of Test
Group (including coordination to make testing a W3C critical
effort.)
* Action Item - Karl Develop proposal as to how to get WGs to
effectively assign resources for testing activities
Framework Documents Status
FPWD published Feb. 1. There has been no feedback since the FPWD. We
also have loose outlines for a) Specification Guidelines and b) P&O
Examples and Techniques. The target date is March 11 for Working
Group publication of the first 4 parts Intro, P&O, P&O examples and
Spec Guidelines.
* Action Item - Karl Put matrix of document family on Working Group
page with links to published versions.
Publication Schedule and Plans
FPWG of first 4 parts (intro, P&O, P&O examples, spec) by beginning of
April. Target for everything (all 7 parts) is September. The target
for the second working draft of all parts is Jan. 2003.
Peter and Mark volunteered to be contributors to the fourth document
(Test Materials.) Karill will be the Editor.
Issues processing
Kirill then discussed his revisions to the Process and Operational
Guidelines document. Kirill revised Guideline 1 according to Lynne
Rosenthalís suggestions. Kirill accepted Lynneís rewrite of Chapter 3
- WG relationship to QA Activity. Karill will cut and paste Lynneís
rewrite of Chapter 3. Lynneís suggestions for the rewrite of
Guideline 4 were incorporated by Karillís reorganization of the
document.
_________________________________________________________________
Meeting minutes - Afternoon
Scribe: Peter Fawcett
Three new observers present since Lunch:
* Nadiah Heninger
* Ian Hickson
* Steve Bratt
Tools:
Olivier started off with a summary of tools for the working group.
First there are some web pages to assist in publishing documents for
the qa web space. These include tools for generating templates for new
web pages, a validator to check that the page is valid for the site
and a link checker. (NOTE: Should I include the urls?, these sites
seem public as will be the email... is that a concern? I have the
various uri's if they are required.)
There is a new page for WG members that contains lots of useful
information. There was some discussion as to wether this document
should be in public or private space, no resolution of the issue was
reached but it was decided to leave it as it is for now.
Finally there is a new search tool for the email archives. This can be
found at [24]http://www.w3.org/Search/Mail/Member/search. The one
trick demonstrated was if you use 'mon year' (ex 'Feb 2002') for the
lookfor: field it will give messages from just that month/year.
[24] http://www.w3.org/Search/Mail/Member/search
Nadia gave a demonstration of the test harness she has been working
on. It uses n3 markup to describe a series of tests and will then
generate a pass/fail/NA web interface with comments on each test case.
Two forms of web interface can be created, one uses frames to have
both the test case and the pass/fail controls in the same window. The
other uses a list of the cases with their pass/fail results in one
window and another window/web application to display the test case.
The results of the tests are output in EARL.
This tool isn't finished yet so it isn't publicly available yet. There
was a suggestion that this tool could be used for validating
checkpoint types guidelines as well.
Issues Processing:
[25]Issue #47: Should test materials be published in TR space? We
revisited this, is it really closed. Decision was Yes. But we need to
clarify why we feel this way. PF was given an action item (ACTION:
A-2002-03-1-6 ) to go through the mail archive and write up a summary
of the justification.
[25] http://www.w3.org/QA/WG/qawg-issues-html.html#x47
This brought up the topic of the rating/endorsement of test materials.
There has been an email thread along this line in the IG mailing list.
The concern seems to be that the WG is claiming that test suites
should be rated by some criteria of goodness. The WG does not feel
that this is the case, what we are proposing is to rate is the level
of conformance to a set of guidelines. It was decided that this issue
related to our already existing issue #44 but that issue #44 was too
general so there is an action item to LH (ACTION: A-2002-03-1-7 ) to
split issue #44 into two new issues. One concerning wether to and how
to rank test materials and one concerning the endorsement of test
material. As for the mail thread, it was decided to continue as it is
and not issue a WG opinion for now.
[26]Issue #44: Should W3C endorse externally produced test suites? MS
pointed out that conformance doesn't mean that a test suite is
perfect, it just means it passed some set of criteria. DD One problem
with conformance is that our guideline wont be ready for 6 months so
we wont be able to hold test materials from current working groups to
what ever future standard we set.
[26] http://www.w3.org/QA/WG/qawg-issues-html.html#x44
LH We aren't able to judge test materials for 'good ness' but we can
comment on or judge test materials on their publishing methods. Is it
public and freely available, does it have a system from handling
errata and so on.
MS We can't check the depth of a test suite because any significancy
complex system will have a nearly infinite number of combinations to
test but we can check test suites against some set of check point
criteria.
Distinction made between High level check, method of publishing, and
low level check, some judgment of the worth of the test materials.
DH/LH It is easier to hold test suites designed with in the WG process
to some standard that it is with test materials designed out side of
the of the WGs.
dd Outside test materials/suites should only be included in matrix if
they agree to be tested for compliance.
Decision was Yes to conformance testing for test materials, No to
Endorsement of test materials and Only include outside test materials
that agree to be tested for conformance.
Break for Coffee.
New Guest: Ian Jacobs
LH proposed postponing discussion of [27]issue #55 till after the WG
has had time to read the new document. All agreed to Thursday March 7
at the usual time. ACTION: A-2002-03-1-8 LH Get bridge for Thursday
(March 7) Telicon.
[27] http://www.w3.org/QA/WG/qawg-issues-html.html#x55
[28]Issue #23: Tests for MAY/SHOULD assertions.
[28] http://www.w3.org/QA/WG/qawg-issues-html.html#x23
MS felt that we should have a way of testing the 'may/should' in the
guidelines.
There was some discussion about the language and the difference
between Priority levels and normative language. For an implementation
to pass with normative language it must pass the MUST tests but not
the MAY or SHOULD tests. With the guideline and checkpoint type
Conformance Levels if a WG commits to a level of conformance they Must
pass all checkpoints up to the required priority level. If a WG
commits to 'AA' then they must pass all Priory 1 and 2 checkpoints.
[29]Issue #56 was opened by MS. Issue being that we need an objective
test suite for testing checkpoints in guidelines document. All agreed
and issue is Closed.
[29] http://www.w3.org/QA/WG/qawg-issues-html.html#x56
[30]Issue #23 was closed once every one agreed on removing all
normative language from the guidelines documents. Also agreeing that
priority levels need to be testable to verify a given level of
conformance.
[30] http://www.w3.org/QA/WG/qawg-issues-html.html#x23
Issue #54: Several structure and content issues about Proc&Ops
Guideline 6. This is a multi part one. it is too long - Not an issue
any longer it's been broken up.
checkpoint 6.1 is not verifiable and shouldn't be there as a
checkpoint - All agree it should be made a note rather than a
checkpoint.
checkpoint 6.2 looks like a duplicate of 6.5, or 6.5 + 6.2 - in any
case, it looks like it's already covered by following checkpoints; -
Language needs to be clarified. These are separate checkpoints, one
dealt with the way to except an outside test suite, the other being
how to review material once accepted. As they are now one appears to
be a subset of the other.
checkpoint 6.7 is similar to 3.2 and 3.3. - WG doesn't think that this
is an issue.
Close Issue.
[31]Issue #52: Does the QA Working Group need a "Process Document"?
[31] http://www.w3.org/QA/WG/qawg-issues-html.html#x52
LH Good beginning would be covering process of handling stuff in
chapter 3 of the Guidelines document.
DD WAI WG doesn't have process document, they do have special alias
mail archive of comments from the group for the record.
KD process document is more for us, our process than for the outside.
LR Agrees.
WG Agrees we need process document.
ACTION: A-2002-03-1-9 KD Claims ownership of process document for
QAWG.
[32]Issue #53: What is the process for handling requests from the WGs?
Postpone resolution of Issue #53 till after we have a draft of the
process document to discuss.
[32] http://www.w3.org/QA/WG/qawg-issues-html.html#x53
WG agrees.
[33]Issue #49: Should there be a global (W3C-standard) license for use
and distribution of test materials?
[33] http://www.w3.org/QA/WG/qawg-issues-html.html#x49
KG pointed out that there is really two licenses involved, one for
documents/suites submitted to a WG from outside and a second for
documents/suites published by the WG.
Should we come up with a template for first type.
Second type should usually use Document License.
dd Takes a stand that we should NOT dictate what license to use.
LH disagrees.
Suggestion to put item to J. Reagel from legal, LH discussed email
discussion he has already had.
PF suggested adding language to justify why WG was suggesting Document
license in most cases but why Software license may be needed.
WG agreed. Issue closed by adding language to guide line to explain
justification for suggestion.
DD Pointed out to be aware of existing pubrules and style guides when
working on our guidelines.
WG decided next Face to Face should be in June 13-14 seemed to be the
only dates available for some people.
ACTION: A-2002-03-1-10 KD To look into possibility of finding place
for Face to Face in Montreal.
Next Telicon: Thurs 3/7 then back to regular schedule on 3/14.
The next meeting will be held in Montreal June 13 and 14.
The meeting was adjourned at 1720.
_________________________________________________________________
[34]Valid XHTML 1.0!
[34] http://validator.w3.org/check/referer
Created Date: 2002-02-21 by [35]Olivier Thereaux
Last modified $Date: 2002/03/06 17:07:16 $ by $Author: kdubost $
--
Karl Dubost / W3C - Conformance Manager
http://www.w3.org/QA/
--- Be Strict To Be Cool! ---
Received on Wednesday, 6 March 2002 14:25:19 UTC