RE: [DRAFT] F2F QA meeting 1st March.

Thanks, Karl!
Looks great to me, except 

- Could you correct my name to Kirill? You use different names for me across the document - I really have only one:)



-----Original Message-----
From: Karl Dubost [mailto:karl@w3.org] 
Sent: Wednesday, March 06, 2002 11:07 AM
To: www-qa-wg@w3.org
Subject: [DRAFT] F2F QA meeting 1st March.


Hi attendees

You have one week to review these minutes ;)

http://www.w3.org/QA/2002/03/01-f2f-minutes

The text version
------------------
    [1]W3C [2]QA

       [1] http://www.w3.org/
       [2] http://www.w3.org/QA/

    Conformance and Quality Assurance

    [3]Action Items· [4]Minutes· [5]Item 3·

      _________________________________________________________________

    Nearby: [6]QA Homepage· [7]Latest News· [8]QA Resources· [9]QA IG·
    [10]QA WG· [11]QA Calendar·

       [6] http://www.w3.org/QA/
       [7] http://www.w3.org/QA/#latest
       [8] http://www.w3.org/QA/#resources
       [9] http://www.w3.org/QA/IG/
      [10] http://www.w3.org/QA/WG/
      [11] http://www.w3.org/QA/Agenda/

Minutes of QA f2f meeting 1st March 2002

    See the [12]Agenda.

      [12] http://www.w3.org/QA/2002/02/F2F-agenda-20020301

Attendees:

      * (LH) Lofton Henderson (CGMO - WG co-chair)
      * (KD) [13]Karl Dubost (W3C, WG co-chair)
      * (LR) Lynne Rosenthal (NIST - IG co-chair)
      * (MS) Mark Skall (NIST)
      * (OT) [14]Olivier Théreaux
      * (DD) [15]Daniel Dardailler (W3C - IG co-chair)
      * (DH) [16]Dominique Hazael-Massieux (W3C - Webmaster)
      * (PF) Peter Fawcett (Real Networks)
      * (KG) Kirill Gavrylyuk (Microsoft)
      * (dd) Dimitris Dimitriadis (Ontologicon)
      * Colas Nahaboo
      * [17]Bert Bos
      * [18]Wendy Chisholm
      * [19]Charles McCathieNeville
      * [20]Susan Lesch
      * Carine Bournez
      * [21]Yvon Lafon

      [13] http://www.w3.org/People/karl/
      [14] http://www.w3.org/People/olivier
      [15] http://www.w3.org/People/danield
      [16] http://www.w3.org/People/dom/
      [17] http://www.w3.org/People/Bos
      [18] http://www.w3.org/People/wendy
      [19] http://www.w3.org/People/Charles/
      [20] http://www.w3.org/People/Lesch/
      [21] http://www.w3.org/People/lafon

Summary of Action Items

      * A-2002-03-1-1 Karl  Clarify proposal about [22]summary of QA IG
        (and WG) mailing lists.
      * A-2002-03-1-2 Olivier and Lofton  Post revised tech plenary
        slides.
      * A-2002-03-1-3 Dimitris  Develop proposal for formation of Test
        Group (including coordination to make testing a W3C critical
        effort.)
      * A-2002-03-1-4 Karl  Develop proposal as to how to get WGs to
        effectively assign resources for testing activities
      * A-2002-03-1-5 Karl and Lofton  Put [23]matrix of document family
        on Working Group page with links to published versions.
      * A-2002-03-1-6 PF Go through mail archives and write up 4-5 top
        justifications for why we don't think test materials should be
        published in TR space.
      * A-2002-03-1-7 LH Split issue #44 into 2 issues. The first dealing
        with the 'ranking' of test materials and the second dealing with
        the 'endorsement of test materials.
      * A-2002-03-1-8 LH Get bridge for Thursday (March 7) Telecon.
      * A-2002-03-1-9 KD Claims ownership of process document for QAWG.
      * A-2002-03-1-10 KD To look into possibility of finding place for
        Face to Face in Montreal.

      [22] http://www.w3.org/QA/2002/02/aweekinqa
      [23] http://www.w3.org/QA/WG/#docs

Meeting minutes - Morning

    Roll Call

Membership Topics

    Rob Lanphier is no longer a member of the Working Group; he is just an
    observer. A discussion ensued concerning the need for more members.
    Although it was agreed that the QA Working Groupís efforts are
    becoming much more visible, the consensus was that having more members
    would allow the Group to maintain our ambitious schedule and large
    amount of deliverables.

Outreach and Education

    Karl Dubost talked about his activity to develop an overview about
    whatís going on the QA mailing list. It will be interesting for people
    (journalist, web community as large, occasional readers) to have an
    abstract of what has been discussed on the mailing list. The goal is
    both to inform people who are not involved in our activity as well as
    to encourage people to join the activity.

    The original proposal was to create a pool of 3 people in charge of
    writing this overview each week, alternatively. A team member of the
    QA activity will review the overview.  It was decided to include the
    Interest Group and any relevant Working Group traffic in the
    overview.  A discussion ensued concerning whether weekly was too often
    to issue the overview.  Monthly was also proposed but that was thought
    not often enough.  It was finally decided to produce the overview
    bi-weekly.  It was decided to try this for 6 months and then
    re-evaluate the utility of the overview.
      * Action Item - Karl  Clarify proposal about summary of QA IG (and
        WG) mailing lists.
      * Action Item - Olivier and Lofton  Post revised tech plenary
        slides.

Liaisons

    The following liaison reports were given:
      * CSS (Lofton)  Good feedback was received on the matrix.  CSS was
        unaware of the QA activity.
      * ATAG (Lofton)  ATAG techniques document is relevant to our
        examples and techniques document.
      * HTML (Lofton)  Lofton made presentation.  They liked the matrix.
      * XML Core (Lofton, Lynne)  Entry in matrix needs to be updated.
        NIST/OASIS test suite transferred to XML/Core.   Tests will be
        revised and fixed.  Test suite will be published as a Working
        Group deliverable.
      * WAI EO (Daniel)  Better way to implement WAI Guidelines by better
        reviews of web sites using the NIST (Skall, Rosenthal) white
        paper.
      * DOM (Dimitris)  Need resources allocated for test materials.

    A discussion then ensued regarding how to get Working Groups to
    allocate enough resources to do an effective job in testing and in
    meeting the QA requirements. There were two alternatives proposed:
     1. A liaison should be identified in each Working Group.  This
        liaison would be responsible for ensuring that the Working Group
        meet all of the QA Groupís requirements (proposed by Mark);
     2. The W3C team should provide liaisons to the Working Group to
        ensure that the Working Groups meet all the QA requirements
        (proposed by Karl);
     3. Coordinate testing to make it a W3C critical effort (proposed by
        Dimitris).

      * Action Item - Dimitris  Develop proposal for formation of Test
        Group (including coordination to make testing a W3C critical
        effort.)
      * Action Item - Karl  Develop proposal as to how to get WGs to
        effectively assign resources for testing activities

Framework Documents Status

    FPWD published Feb. 1. There has been no feedback since the FPWD.  We
    also have loose outlines for a) Specification Guidelines and b) P&O
    Examples and Techniques.  The target date is March 11 for Working
    Group publication of the first 4 parts  Intro, P&O, P&O examples and
    Spec Guidelines.
      * Action Item - Karl  Put matrix of document family on Working Group
        page with links to published versions.

Publication Schedule and Plans

    FPWG of first 4 parts (intro, P&O, P&O examples, spec) by beginning of
    April.  Target for everything (all 7 parts) is September.  The target
    for the second working draft of all parts is Jan. 2003.

    Peter and Mark volunteered to be contributors to the fourth document
    (Test Materials.) Karill will be the Editor.

Issues processing

    Kirill then discussed his revisions to the Process and Operational
    Guidelines document.  Kirill revised Guideline 1 according to Lynne
    Rosenthalís suggestions.  Kirill accepted Lynneís rewrite of Chapter 3
    - WG relationship to QA Activity.  Karill will cut and paste Lynneís
    rewrite of Chapter 3.  Lynneís suggestions for the rewrite of
    Guideline 4 were incorporated by Karillís reorganization of the
    document.

      _________________________________________________________________

Meeting minutes - Afternoon

    Scribe: Peter Fawcett

    Three new observers present since Lunch:
      * Nadiah Heninger
      * Ian Hickson
      * Steve Bratt

Tools:

    Olivier started off with a summary of tools for the working group.

    First there are some web pages to assist in publishing documents for
    the qa web space. These include tools for generating templates for new
    web pages, a validator to check that the page is valid for the site
    and a link checker. (NOTE: Should I include the urls?, these sites
    seem public as will be the email... is that a concern? I have the
    various uri's if they are required.)

    There is a new page for WG members that contains lots of useful
    information. There was some discussion as to wether this document
    should be in public or private space, no resolution of the issue was
    reached but it was decided to leave it as it is for now.

    Finally there is a new search tool for the email archives. This can be
    found at [24]http://www.w3.org/Search/Mail/Member/search. The one
    trick demonstrated was if you use 'mon year' (ex 'Feb 2002') for the
    lookfor: field it will give messages from just that month/year.

      [24] http://www.w3.org/Search/Mail/Member/search

    Nadia gave a demonstration of the test harness she has been working
    on. It uses n3 markup to describe a series of tests and will then
    generate a pass/fail/NA web interface with comments on each test case.
    Two forms of web interface can be created, one uses frames to have
    both the test case and the pass/fail controls in the same window. The
    other uses a list of the cases with their pass/fail results in one
    window and another window/web application to display the test case.

    The results of the tests are output in EARL.

    This tool isn't finished yet so it isn't publicly available yet. There
    was a suggestion that this tool could be used for validating
    checkpoint types guidelines as well.

    Issues Processing:

    [25]Issue #47: Should test materials be published in TR space? We
    revisited this, is it really closed. Decision was Yes. But we need to
    clarify why we feel this way. PF was given an action item (ACTION:
    A-2002-03-1-6 ) to go through the mail archive and write up a summary
    of the justification.

      [25] http://www.w3.org/QA/WG/qawg-issues-html.html#x47

    This brought up the topic of the rating/endorsement of test materials.
    There has been an email thread along this line in the IG mailing list.
    The concern seems to be that the WG is claiming that test suites
    should be rated by some criteria of goodness. The WG does not feel
    that this is the case, what we are proposing is to rate is the level
    of conformance to a set of guidelines. It was decided that this issue
    related to our already existing issue #44 but that issue #44 was too
    general so there is an action item to LH (ACTION: A-2002-03-1-7 ) to
    split issue #44 into two new issues. One concerning wether to and how
    to rank test materials and one concerning the endorsement of test
    material. As for the mail thread, it was decided to continue as it is
    and not issue a WG opinion for now.

    [26]Issue #44: Should W3C endorse externally produced test suites? MS
    pointed out that conformance doesn't mean that a test suite is
    perfect, it just means it passed some set of criteria. DD One problem
    with conformance is that our guideline wont be ready for 6 months so
    we wont be able to hold test materials from current working groups to
    what ever future standard we set.

      [26] http://www.w3.org/QA/WG/qawg-issues-html.html#x44

    LH We aren't able to judge test materials for 'good ness' but we can
    comment on or judge test materials on their publishing methods. Is it
    public and freely available, does it have a system from handling
    errata and so on.

    MS We can't check the depth of a test suite because any significancy
    complex system will have a nearly infinite number of combinations to
    test but we can check test suites against some set of check point
    criteria.

    Distinction made between High level check, method of publishing, and
    low level check, some judgment of the worth of the test materials.

    DH/LH It is easier to hold test suites designed with in the WG process
    to some standard that it is with test materials designed out side of
    the of the WGs.

    dd Outside test materials/suites should only be included in matrix if
    they agree to be tested for compliance.

    Decision was Yes to conformance testing for test materials, No to
    Endorsement of test materials and Only include outside test materials
    that agree to be tested for conformance.

    Break for Coffee.

    New Guest: Ian Jacobs

    LH proposed postponing discussion of [27]issue #55 till after the WG
    has had time to read the new document. All agreed to Thursday March 7
    at the usual time. ACTION: A-2002-03-1-8 LH Get bridge for Thursday
    (March 7) Telicon.

      [27] http://www.w3.org/QA/WG/qawg-issues-html.html#x55

    [28]Issue #23: Tests for MAY/SHOULD assertions.

      [28] http://www.w3.org/QA/WG/qawg-issues-html.html#x23

    MS felt that we should have a way of testing the 'may/should' in the
    guidelines.

    There was some discussion about the language and the difference
    between Priority levels and normative language. For an implementation
    to pass with normative language it must pass the MUST tests but not
    the MAY or SHOULD tests. With the guideline and checkpoint type
    Conformance Levels if a WG commits to a level of conformance they Must
    pass all checkpoints up to the required priority level. If a WG
    commits to 'AA' then they must pass all Priory 1 and 2 checkpoints.

    [29]Issue #56 was opened by MS. Issue being that we need an objective
    test suite for testing checkpoints in guidelines document. All agreed
    and issue is Closed.

      [29] http://www.w3.org/QA/WG/qawg-issues-html.html#x56

    [30]Issue #23 was closed once every one agreed on removing all
    normative language from the guidelines documents. Also agreeing that
    priority levels need to be testable to verify a given level of
    conformance.

      [30] http://www.w3.org/QA/WG/qawg-issues-html.html#x23

    Issue #54: Several structure and content issues about Proc&Ops
    Guideline 6. This is a multi part one. it is too long - Not an issue
    any longer it's been broken up.

    checkpoint 6.1 is not verifiable and shouldn't be there as a
    checkpoint - All agree it should be made a note rather than a
    checkpoint.

    checkpoint 6.2 looks like a duplicate of 6.5, or 6.5 + 6.2 - in any
    case, it looks like it's already covered by following checkpoints; -
    Language needs to be clarified. These are separate checkpoints, one
    dealt with the way to except an outside test suite, the other being
    how to review material once accepted. As they are now one appears to
    be a subset of the other.

    checkpoint 6.7 is similar to 3.2 and 3.3. - WG doesn't think that this
    is an issue.

    Close Issue.

    [31]Issue #52: Does the QA Working Group need a "Process Document"?

      [31] http://www.w3.org/QA/WG/qawg-issues-html.html#x52

    LH Good beginning would be covering process of handling stuff in
    chapter 3 of the Guidelines document.

    DD WAI WG doesn't have process document, they do have special alias
    mail archive of comments from the group for the record.

    KD process document is more for us, our process than for the outside.

    LR Agrees.

    WG Agrees we need process document.

    ACTION: A-2002-03-1-9 KD Claims ownership of process document for
    QAWG.

    [32]Issue #53: What is the process for handling requests from the WGs?
    Postpone resolution of Issue #53 till after we have a draft of the
    process document to discuss.

      [32] http://www.w3.org/QA/WG/qawg-issues-html.html#x53

    WG agrees.

    [33]Issue #49: Should there be a global (W3C-standard) license for use
    and distribution of test materials?

      [33] http://www.w3.org/QA/WG/qawg-issues-html.html#x49

    KG pointed out that there is really two licenses involved, one for
    documents/suites submitted to a WG from outside and a second for
    documents/suites published by the WG.

    Should we come up with a template for first type.

    Second type should usually use Document License.

    dd Takes a stand that we should NOT dictate what license to use.

    LH disagrees.

    Suggestion to put item to J. Reagel from legal, LH discussed email
    discussion he has already had.

    PF suggested adding language to justify why WG was suggesting Document
    license in most cases but why Software license may be needed.

    WG agreed. Issue closed by adding language to guide line to explain
    justification for suggestion.

    DD Pointed out to be aware of existing pubrules and style guides when
    working on our guidelines.

    WG decided next Face to Face should be in June 13-14 seemed to be the
    only dates available for some people.

    ACTION: A-2002-03-1-10 KD To look into possibility of finding place
    for Face to Face in Montreal.

    Next Telicon: Thurs 3/7 then back to regular schedule on 3/14.

    The next meeting will be held in Montreal June 13 and 14.

    The meeting was adjourned at 1720.

      _________________________________________________________________

    [34]Valid XHTML 1.0!

      [34] http://validator.w3.org/check/referer

    Created Date: 2002-02-21 by [35]Olivier Thereaux
    Last modified $Date: 2002/03/06 17:07:16 $ by $Author: kdubost $


-- 
Karl Dubost / W3C - Conformance Manager
           http://www.w3.org/QA/

      --- Be Strict To Be Cool! ---

Received on Wednesday, 13 March 2002 02:03:16 UTC