W3C home > Mailing lists > Public > www-qa-wg@w3.org > March 2003

Re: Fwd: Draft Minutes from Friday PM

From: Lofton Henderson <lofton@rockynet.com>
Date: Wed, 12 Mar 2003 19:14:16 -0700
Message-Id: <5.1.0.14.2.20030312190844.02bf2c10@rockynet.com>
To: pfawcett <pfawcett@speakeasy.org>
Cc: www-qa-wg@w3.org

One amendment for the Fri PM minutes (maybe more later)...

At 09:02 AM 3/10/03 -0800, pfawcett wrote:

>Begin forwarded message:
>
>>From: pfawcett <pfawcett@speakeasy.org>
>>Date: Sun Mar 9, 2003  8:21:13  AM US/Pacific
>>To: Olivier Thereaux <ot@w3.org>
>>Subject: Draft Minutes from Friday PM
>>
>>QA Working Group Face To Face
>>Friday, 7-March-2003 Afternoon session.
>>--
>>[...]
>>Summary of New Action Items:

There actually was one action item that I remember, and I have confirmed it 
with Peter.  Not only is Peter going to join "TTF Charter" as a co-editor, 
but he is going to work on a next draft that incorporates our detailed 
deliverables discussions and strategies from the f2f.  So

AI-20030307-x   Peter   Next draft TTF Charter with detailed deliverables 
and strategies from the f2f  due: 20030412

-Lofton.



>>Agenda: http://www.w3.org/QA/2003/03/f2f
>>
>>Previous Telcon Minutes: 
>>http://lists.w3.org/Archives/Public/www-qa-wg/2003Mar/0011.html
>>
>>
>>Minutes:
>>
>>Framework Completion Plan:
>>
>>         TestGL
>>                 - 3/10 mon telcon will be for TestGL.
>>                 - Publish 2nd working draft around 20th. Likely wont 
>> make that date.
>>                 Will we need another telcon other than monday. likely yes...
>>                 - Second TestGL telcon scheduled for 24th march.
>>
>>                 - The 17th telcon will focus on OpsGL and SpecGL.
>>
>>                 - If we are not done with issues for Test on the 24th we 
>> will need to schedule
>>                 another telcon at a time other than the regular time.
>>
>>                 - Publish another Working Draft of TestGL two weeks 
>> after the Second
>>                         Telcon (around april 7).
>>                 - The go to last call with TestGL by 6/4
>>
>>                 - TestGl is running about 5 months behind.
>>
>>         Issues processing on Ops and Spec.
>>                 - We should try and be done with issue processing by 
>> around the 26th-27th.
>>                 Then we prepare for next cycle.
>>                 - what is next cycle? 2nd Last call or CR.
>>                 - one concern is that we may be rejected if we make too 
>> many changes...
>>
>>                 - some comments about the organization of ops and spec. 
>> Loften
>>                 mentioned a desire to do some re-org but likely for V2.
>>                 Dominique, ops has to take into account time line of 
>> process and may need
>>                 some reorganization as well...
>>
>>                 -  If we are going to CR we need to be very exact and 
>> careful about our
>>                 issue processing. Send responses to issue submitters, 
>> document all issues
>>                 and resolutions. Document issues that we have decided 
>> are not issues and
>>                 so on.
>>                 - Also need to prepare documents and do actual editing.
>>                 - Take about 3 weeks for the two documents. Taking us to 
>> Mid may.
>>
>>                 - Is there any other issues for going to CR.
>>                         Need CR Call. Meeting to cover CR.
>>                 - Once we go to CR, it will be 3 to 6 months (give or 
>> take) till we can
>>                         finish CR. Part of the exit criteria is to have 
>> two implementations.
>>                 - Our current criteria is two implementations that 
>> conform to all p1 and p2
>>                         checkpoints.
>>
>>                 - That means that it will be late 2003 give or take to 
>> get out of CR
>>                 for OpsGL and SpecGL. TestGL is running behind and 
>> likely wont be
>>                 getting out of CR till early mid 2004.
>>
>>                 - Still not positive that the mid may date will be CR or 
>> 2nd Last Call.
>>
>>         - What does that put on our agenda for Crete.
>>                 - Tests for last call.
>>                 - What ever is going on with OpsGL/SpecGL
>>                 - Rechartering.
>>
>>         - We still have not determined if our final goal is Rec or Note.
>>                 it seems that people expect us to go to Rec.
>>
>>         Since we are dealing with process, the advisory committee may 
>> have something to
>>                 say about our work. At what point do we start thinking 
>> about integrating some
>>                 of these Guidelines in to the bigger process. For 
>> example pub rules.
>>
>>         - Karl pointed out that we should comply to our own guidelines. 
>> We need to at
>>                 least address the guidelines and determine if they are 
>> applicable or not.
>>                 This should happen before CR.
>>
>>                 - There are issues for what our 'test materials' are. 
>>  From the break out group
>>                 on Thursday, we had determined that our checklists are 
>> our 'test materials' but
>>                 it is some what different than conventional 'test 
>> materials'.
>>
>>                 - Karl would like to try to apply our own criteria to 
>> our selves. We should
>>                 make an attempt to move our own process/orginization in 
>> the direction we
>>                 hold others too.
>>
>>                 - Loften - of we get our process doc and an addendum to 
>> the charter we can
>>                 start passing a number of the checkpoints that we 
>> currently fail.
>>
>>                 - This should be done before we are done with issues 
>> processing.
>>
>>                 - If we have the process document and the charter are 
>> updated we should
>>                 re-evaluate our selves.
>>                 Put an update on this on one of the April Telcon Agendas.
>>
>>         - Given all this the earliest that we can get to Rec is late 
>> this year.
>>                 Test later, probably early next year. Perhaps around 
>> next plenary.
>>
>>TTF/QAWG future and re-charter.
>>
>>         - Loften had heard a comment that TTF may violate the w3c 
>> process document.
>>         however Loften claims that section 4.1 does allow us to do this 
>> with an
>>         informal charter.
>>         and as an aside to the above comments about conforming to our own
>>         checkpoints, our current draft charter does not comply with our own
>>         OpsGL in that we do not mention qa resources....
>>
>>         - QAWG2 will likely be mostly TTF and outreach mostly.
>>
>>         - we have draft charter for TTF. We have some deliverables, 
>> scope and success
>>         criteria.
>>
>>         - what do we need to do to the draft TTF charter in order to say 
>> that we
>>         are done with it?
>>
>>         - Karl suggested to see if there is interest in the next 
>> charter. Let people
>>         know that it's almost done but we don't currently have staffing 
>> resources
>>         and is any one interested in giving us some help?
>>
>>         - Deliverables can be included in task force charter as it's not 
>> as formal
>>         as a wg charter. Some of these may migrate to the WG re-charter.
>>
>>         - Milestones is one part that needs to be filled in with 
>> uncertain dates
>>         as we don't have staffing yet.
>>
>>         - List of possible deliverables for TTF. (from lofton with some 
>> additions from
>>                 others, largely from out reach sessions )
>>                 - TS Management framework. (Tool)
>>                         say take VoiceML testing tool that was demoed to 
>> us yesterday,
>>                         and generalize it for reuse.
>>                 - Test Assertion prototype/specification or markup. (Spec)
>>                         what ever that becomes should be moved toward 
>> incorporation in
>>                         style guide. We do not want to end up with 
>> multiple markup
>>                         guides in multiple places.
>>                 - Binary serialized infoset and comparator. (tool)
>>                 - Test Authoring Principles - (document). (from this 
>> morning discussion).
>>                         have samples from css already.
>>                 - Test suite and Test Materials assessment and 
>> description. (document/research)
>>                         an extension to the matrix perhaps.
>>                         describe how it is done in the w3c currently.
>>                         stay away from rating. Only description and 
>> assessment.
>>                 - Reporting/Earl/tools framework. (Tool)
>>                 - Other new templates. (document)
>>                 - Test case description language /metadata for testcases 
>> (spec)
>>
>>         - who will build these tools.
>>
>>         - Patrick - Sun is working on Tools to do some test assertion 
>> analysis
>>         of specifications. If possible he may be able to leverage it and
>>         move it into back into our group/w3c.
>>
>>         - Likely two incarnations of such a tool, one for xhtml markup with
>>         classes, the other for xml like xmlspec.
>>
>>         - Susan Lesch, don't worry too much about manual of style as it 
>> can be changed,
>>         still a flexible document.
>>
>>         - Once TTF Charter document is done and include above 
>> deliverables and then
>>         send out invitations for participation to help develop these tools.
>>
>>         - If call for participation is not met favorably, we don't 
>> really loose anything.
>>         We will just be in the same boat that we are already in.
>>
>>         Also even if we are over reaching and can't really build some of 
>> these tools,
>>         the templates have proven useful and people like them, so more 
>> of them
>>         could be easy fast way to get some support out there and give us 
>> a good affect
>>         with out a lot of effort.
>>
>>         - the XSLT group had done a very good job creating all aspects 
>> of a test
>>         framework including contribution and review of test cases. This 
>> is one thing
>>         that every group has to do so if we came up with a review 
>> process template
>>         it could well help many groups.
>>
>>         - A test challenge process template is another one that every WG 
>> needs.
>>
>>         - There is a lot out there that is reusable, we can likely 
>> steal, generalize and
>>         reuse what is good.
>>
>>         Patrick, Concepts/catigories from Sun in process and tools to 
>> help process.
>>         - tools for Program management collaboration,
>>         - tools for test case generation
>>         - test execution framework
>>         - documentation for test harness.
>>         - deployment, maintenance, update
>>
>>         - Another tool could be XSLT tools that have a selection 
>> mechanism that can
>>         determine what tests are valid for what profile or implementation.
>>
>>         - Now that we have a notion of some of the possible 
>> deliverables: we can update
>>         the scope and deliverables sections of the draft charter.
>>         - Milestones can then also be started, at least with some of the 
>> easier tools.
>>         - One milestone would be a solicitation from other WG's about 
>> what the priority
>>         of tools to build should be, a determination of what tools are 
>> more important.
>>         what do groups need first.
>>         - Loften - we can also grab the easier ones that are more 
>> obvious while waiting
>>         for comments on priorities.
>>         - People may contribute more or other ideas that we have not 
>> thought of.
>>
>>         - Patrick agreed to be co-editor of document.
>>
>>         - Summary:
>>                 Patrick will be co-editor.
>>                 Came up with list of first possible deliverables.
>>                 Seem to have enough resources to at least start on some 
>> of the easier
>>                         tools
>>                 Send out call for participation. and call for 
>> comments/priorities/ideas.
>>
>>         - We have been talking about this for a long time, lets see if 
>> we can start
>>                 on a few of the small things.
>>
>>Back to TestGL:
>>         - Loften's comments from his last review of the document:
>>                 - Intro needs to be updated with new intro that matches 
>> the headers from
>>                 SpecGL/OpsGL and rework it as appropriate for TestGL.
>>                 Editor should do this likely. It also needs a lot of 
>> work. See the next
>>                 few comments on specific issues.
>>
>>                 - Chapter 1 (intro) 1 second comment in paragraph 3 is 
>> likely out of scope - WG agreed.
>>
>>                 - Chapter 1 (intro) 1 paragraph 4 / bullet list/5 the is 
>> really hard to read. and understand.
>>                 Lofton suggested that first sentence of 5 could be first 
>> sentence with
>>                 bullet list and nothing else.
>>
>>                 - Chapter 1 Section 1.1 - Really confusing and hard to read.
>>
>>                 - In this document comply, compliance is used. Should 
>> use conform/conformance
>>                 instead. We do not test compliance, we test conformance.
>>
>>                 - 2nd bullet in 1.1 list, lofton disagrees with this 
>> point, it's false.
>>                 WG agrees.
>>
>>                 - 1.5 "test area" what is it, better definition? or 
>> remove it.
>>                 also glossary should be moved to end as the rest of the 
>> documents.
>>
>>                 - Many checkpoints have multiple sentence  long prose. 
>> Needs to be shorter
>>                 and terser. Many of them will become rational, further 
>> descriptions or
>>                 fulfillment criteria.
>>
>>                 - others are just long and not concise like 5.3 and 7.2.
>>
>>                 - Need to reformat as SpecGL/OpsGL.
>>                 Title, Conformance Requirements, This will be more than 
>> a formatting
>>                 excise. Lead editor will call on co-editors for help as 
>> needed.
>>                 Need to first do basic restructure.
>>
>>                 - Checkpoint 1.3 - The wording is bad. What we believe 
>> that we mean is that
>>                 dimensions of variability should be taken into account.
>>                 By imposing grouping, it forces it to be a done in a 
>> single way.
>>                 It currently focuses on asserts. It should likely focus 
>> on test cases
>>                 so that tests can be selected based on dimensions of 
>> variability.
>>
>>                 - Much of the issues with this current document is that 
>> it focus on the
>>                 "How to write a test suite" not "what a test suite 
>> should be"
>>
>>Move to adjourn.
>>Agreed.
>
Received on Wednesday, 12 March 2003 21:13:33 GMT

This archive was generated by hypermail 2.2.0 + w3c-0.30 : Thursday, 9 June 2005 12:13:13 GMT