Fwd: Draft Minutes from Friday PM

Begin forwarded message:

> From: pfawcett <pfawcett@speakeasy.org>
> Date: Sun Mar 9, 2003  8:21:13  AM US/Pacific
> To: Olivier Thereaux <ot@w3.org>
> Subject: Draft Minutes from Friday PM
>
> QA Working Group Face To Face
> Friday, 7-March-2003 Afternoon session.
> --
> Scribe: Peter Fawcett
>
> Attendees:
> (KD) Karl Dubost (W3C, WG co-chair)
> (PF) Peter Fawcett (RealNetworks)
> (DH) Dominique Hazael-Massieux (W3C - Webmaster)
> (LH) Lofton Henderson (CGMO - WG co-chair)
> (SM) Sandra Martinez (NIST)
> (PC) Patrick Curran (Sun)
> (LR) Lynne Rosenthal (NIST - IG co-chair)
> (MS) Mark Skall (NIST)
>
> Regrets:
> (dd) Dimitris Dimitriadis (Ontologicon)
> (KG) Kirill Gavrylyuk (Microsoft)
>
> Guest:
> Susan Lesch.
>
> Summary of New Action Items:
>
>
> Agenda: http://www.w3.org/QA/2003/03/f2f
>
> Previous Telcon Minutes: 
> http://lists.w3.org/Archives/Public/www-qa-wg/2003Mar/0011.html
>
>
> Minutes:
>
> Framework Completion Plan:
> 	
> 	TestGL
> 		- 3/10 mon telcon will be for TestGL.
> 		- Publish 2nd working draft around 20th. Likely wont make that date.
> 		Will we need another telcon other than monday. likely yes...
> 		- Second TestGL telcon scheduled for 24th march.
> 		
> 		- The 17th telcon will focus on OpsGL and SpecGL.
> 		
> 		- If we are not done with issues for Test on the 24th we will need 
> to schedule
> 		another telcon at a time other than the regular time.
> 		
> 		- Publish another Working Draft of TestGL two weeks after the Second
> 			Telcon (around april 7).
> 		- The go to last call with TestGL by 6/4
> 		
> 		- TestGl is running about 5 months behind.
> 	
> 	Issues processing on Ops and Spec.
> 		- We should try and be done with issue processing by around the 
> 26th-27th.
> 		Then we prepare for next cycle.
> 		- what is next cycle? 2nd Last call or CR.
> 		- one concern is that we may be rejected if we make too many 
> changes...
> 		
> 		- some comments about the organization of ops and spec. Loften
> 		mentioned a desire to do some re-org but likely for V2.
> 		Dominique, ops has to take into account time line of process and may 
> need
> 		some reorganization as well...
> 		
> 		-  If we are going to CR we need to be very exact and careful about 
> our
> 		issue processing. Send responses to issue submitters, document all 
> issues
> 		and resolutions. Document issues that we have decided are not issues 
> and
> 		so on.
> 		- Also need to prepare documents and do actual editing.
> 		- Take about 3 weeks for the two documents. Taking us to Mid may.
> 	
> 		- Is there any other issues for going to CR.
> 			Need CR Call. Meeting to cover CR.
> 		- Once we go to CR, it will be 3 to 6 months (give or take) till we 
> can
> 			finish CR. Part of the exit criteria is to have two implementations.
> 		- Our current criteria is two implementations that conform to all p1 
> and p2
> 			checkpoints.
> 		
> 		- That means that it will be late 2003 give or take to get out of CR
> 		for OpsGL and SpecGL. TestGL is running behind and likely wont be
> 		getting out of CR till early mid 2004.
> 		
> 		- Still not positive that the mid may date will be CR or 2nd Last 
> Call.
> 		
> 	- What does that put on our agenda for Crete.
> 		- Tests for last call.
> 		- What ever is going on with OpsGL/SpecGL
> 		- Rechartering.
> 		
> 	- We still have not determined if our final goal is Rec or Note.
> 		it seems that people expect us to go to Rec.
> 	
> 	Since we are dealing with process, the advisory committee may have 
> something to
> 		say about our work. At what point do we start thinking about 
> integrating some
> 		of these Guidelines in to the bigger process. For example pub rules.
> 		
> 	- Karl pointed out that we should comply to our own guidelines. We 
> need to at
> 		least address the guidelines and determine if they are applicable or 
> not.
> 		This should happen before CR.
> 		
> 		- There are issues for what our 'test materials' are. From the break 
> out group
> 		on Thursday, we had determined that our checklists are our 'test 
> materials' but
> 		it is some what different than conventional 'test materials'.
> 		
> 		- Karl would like to try to apply our own criteria to our selves. We 
> should
> 		make an attempt to move our own process/orginization in the 
> direction we
> 		hold others too.
> 		
> 		- Loften - of we get our process doc and an addendum to the charter 
> we can
> 		start passing a number of the checkpoints that we currently fail.
> 		
> 		- This should be done before we are done with issues processing.
> 			
> 		- If we have the process document and the charter are updated we 
> should
> 		re-evaluate our selves.
> 		Put an update on this on one of the April Telcon Agendas.
> 	
> 	- Given all this the earliest that we can get to Rec is late this 
> year.
> 		Test later, probably early next year. Perhaps around next plenary.
>
> TTF/QAWG future and re-charter.
> 	
> 	- Loften had heard a comment that TTF may violate the w3c process 
> document.
> 	however Loften claims that section 4.1 does allow us to do this with 
> an
> 	informal charter.
> 	and as an aside to the above comments about conforming to our own
> 	checkpoints, our current draft charter does not comply with our own
> 	OpsGL in that we do not mention qa resources....
> 	
> 	- QAWG2 will likely be mostly TTF and outreach mostly.
> 	
> 	- we have draft charter for TTF. We have some deliverables, scope and 
> success
> 	criteria.
> 	
> 	- what do we need to do to the draft TTF charter in order to say that 
> we
> 	are done with it?
> 	
> 	- Karl suggested to see if there is interest in the next charter. Let 
> people
> 	know that it's almost done but we don't currently have staffing 
> resources
> 	and is any one interested in giving us some help?
> 	
> 	- Deliverables can be included in task force charter as it's not as 
> formal
> 	as a wg charter. Some of these may migrate to the WG re-charter.
>
> 	- Milestones is one part that needs to be filled in with uncertain 
> dates
> 	as we don't have staffing yet.
> 	
> 	- List of possible deliverables for TTF. (from lofton with some 
> additions from
> 		others, largely from out reach sessions )
> 		- TS Management framework. (Tool)
> 			say take VoiceML testing tool that was demoed to us yesterday,
> 			and generalize it for reuse.
> 		- Test Assertion prototype/specification or markup. (Spec)
> 			what ever that becomes should be moved toward incorporation in
> 			style guide. We do not want to end up with multiple markup
> 			guides in multiple places.
> 		- Binary serialized infoset and comparator. (tool)
> 		- Test Authoring Principles - (document). (from this morning 
> discussion).
> 			have samples from css already.
> 		- Test suite and Test Materials assessment and description. 
> (document/research)
> 			an extension to the matrix perhaps.
> 			describe how it is done in the w3c currently.
> 			stay away from rating. Only description and assessment.
> 		- Reporting/Earl/tools framework. (Tool)
> 		- Other new templates. (document)
> 		- Test case description language /metadata for testcases (spec)
>
> 	- who will build these tools.
> 	
> 	- Patrick - Sun is working on Tools to do some test assertion analysis
> 	of specifications. If possible he may be able to leverage it and
> 	move it into back into our group/w3c.
> 	
> 	- Likely two incarnations of such a tool, one for xhtml markup with
> 	classes, the other for xml like xmlspec.
>
> 	- Susan Lesch, don't worry too much about manual of style as it can 
> be changed,
> 	still a flexible document.
> 	
> 	- Once TTF Charter document is done and include above deliverables 
> and then
> 	send out invitations for participation to help develop these tools.
> 	
> 	- If call for participation is not met favorably, we don't really 
> loose anything.
> 	We will just be in the same boat that we are already in.
> 	
> 	Also even if we are over reaching and can't really build some of 
> these tools,
> 	the templates have proven useful and people like them, so more of them
> 	could be easy fast way to get some support out there and give us a 
> good affect
> 	with out a lot of effort.
> 	
> 	- the XSLT group had done a very good job creating all aspects of a 
> test
> 	framework including contribution and review of test cases. This is 
> one thing
> 	that every group has to do so if we came up with a review process 
> template
> 	it could well help many groups.
> 	
> 	- A test challenge process template is another one that every WG 
> needs.
> 	
> 	- There is a lot out there that is reusable, we can likely steal, 
> generalize and
> 	reuse what is good.
> 		
> 	Patrick, Concepts/catigories from Sun in process and tools to help 
> process.
> 	- tools for Program management collaboration,
> 	- tools for test case generation
> 	- test execution framework
> 	- documentation for test harness.
> 	- deployment, maintenance, update
>
> 	- Another tool could be XSLT tools that have a selection mechanism 
> that can
> 	determine what tests are valid for what profile or implementation.
> 	
> 	- Now that we have a notion of some of the possible deliverables: we 
> can update
> 	the scope and deliverables sections of the draft charter.
> 	- Milestones can then also be started, at least with some of the 
> easier tools.
> 	- One milestone would be a solicitation from other WG's about what 
> the priority
> 	of tools to build should be, a determination of what tools are more 
> important.
> 	what do groups need first.
> 	- Loften - we can also grab the easier ones that are more obvious 
> while waiting
> 	for comments on priorities.
> 	- People may contribute more or other ideas that we have not thought 
> of.
>
> 	- Patrick agreed to be co-editor of document.
> 	
> 	- Summary:
> 		Patrick will be co-editor.
> 		Came up with list of first possible deliverables.
> 		Seem to have enough resources to at least start on some of the easier
> 			tools
> 		Send out call for participation. and call for 
> comments/priorities/ideas.
> 	
> 	- We have been talking about this for a long time, lets see if we can 
> start
> 		on a few of the small things.
>
> Back to TestGL:
> 	- Loften's comments from his last review of the document:
> 		- Intro needs to be updated with new intro that matches the headers 
> from
> 		SpecGL/OpsGL and rework it as appropriate for TestGL.
> 		Editor should do this likely. It also needs a lot of work. See the 
> next
> 		few comments on specific issues.
> 		
> 		- Chapter 1 (intro) 1 second comment in paragraph 3 is likely out of 
> scope - WG agreed.
> 		
> 		- Chapter 1 (intro) 1 paragraph 4 / bullet list/5 the is really hard 
> to read. and understand.
> 		Lofton suggested that first sentence of 5 could be first sentence 
> with
> 		bullet list and nothing else.
> 		
> 		- Chapter 1 Section 1.1 - Really confusing and hard to read.
> 		
> 		- In this document comply, compliance is used. Should use 
> conform/conformance
> 		instead. We do not test compliance, we test conformance.
> 		
> 		- 2nd bullet in 1.1 list, lofton disagrees with this point, it's 
> false.
> 		WG agrees.
> 		
> 		- 1.5 "test area" what is it, better definition? or remove it.
> 		also glossary should be moved to end as the rest of the documents.
> 		
> 		- Many checkpoints have multiple sentence  long prose. Needs to be 
> shorter
> 		and terser. Many of them will become rational, further descriptions 
> or
> 		fulfillment criteria.
> 		
> 		- others are just long and not concise like 5.3 and 7.2.
> 		
> 		- Need to reformat as SpecGL/OpsGL.
> 		Title, Conformance Requirements, This will be more than a formatting
> 		excise. Lead editor will call on co-editors for help as needed.
> 		Need to first do basic restructure.
> 		
> 		- Checkpoint 1.3 - The wording is bad. What we believe that we mean 
> is that
> 		dimensions of variability should be taken into account.
> 		By imposing grouping, it forces it to be a done in a single way.
> 		It currently focuses on asserts. It should likely focus on test cases
> 		so that tests can be selected based on dimensions of variability.
> 		
> 		- Much of the issues with this current document is that it focus on 
> the
> 		"How to write a test suite" not "what a test suite should be"
> 				
> Move to adjourn.
> Agreed.
>

Received on Monday, 10 March 2003 15:17:07 UTC