Final Minutes of QA WG meeting 2003-09-15

QA Working Group Teleconference
Monday, 15-September-2003
--
Scribe:Sandra I. Martinez

(PC) Patrick Curran (Sun Microsystems)
(dd) Dimitris Dimitriadis (Ontologicon)
(KD) Karl Dubost (W3C, WG co-chair)
(MS) Mark Skall (NIST)
(LR) Lynne Rosenthal (NIST - IG co-chair)
(DH) Dominique Hazaël-Massieux (W3C)
(SM) Sandra Martinez (NIST)
(PF) Peter Fawcett (RealNetworks)


Regrets:
(LH) Lofton Henderson (CGMO - WG co-chair)


Absent:
(VV) Vanitha Venkatraman (Sun Microsystems)
(KG) Kirill Gavrylyuk (Microsoft)
(AT) Andrew Thackrah (Open Group)



Summary of New Action Items:
AI-20030915-1: Lynne to draft response to SpecGL comments.
AI-20030915-2: Lynne to find definition of atomic.

Agenda: http://lists.w3.org/Archives/Public/www-qa-wg/2003Sep/0060.html
Previous Telcon Minutes: 
http://lists.w3.org/Archives/Public/www-qa-wg/2003Sep/0059.html

Minutes:
1.) roll call 11am EDT, membership
2.) 2.) Any routine business
       [- overdue Action Items ] [1]
3.) OpsGL CR Implementation Plan
Document published [2]
Implementation Plan for CR [3]

(KD) - I had contacted different WGs to invite them to implement the QA Ops 
Guidelines. SVG, DOM, OWL and WSD groups had decided to put the request in 
their meeting agenda and some of them had expressed not having resources to 
performed the task.
(PC): Are people receptive?
(KD):I have being taking the approach of explaining that nothing is 
mandatory, they are not force to do it. If there is something that they 
don't like they are encourage to tell us. I am trying to give encouragement 
to people to implement.
(PC):I have a concern that it will sit there and no one will pay attention. 
It is unfortunate; we are trying to focus on areas that will benefit.
(KD): This is our first time - we explained, we didn't try to push it further.
(MS): It is too early for them to accept with open hand. They see it as an 
additional burden.
(DH):I think it is a good thing is a challenge for us and we need to push it.
(LR): This is only OpsGl just wait until they get SpecGl and TestGL.
(KD) SpecGL can be very difficult. Yes, we have a challenge.



4.) SpecGL
- WD Published [4]
- -Alex Rousskov's Comment about security [5]
- Is it something similar to the Jon Gunderson's comments?
o (DH): I think security is not necessary in this document; it is out of 
scope of the SpecGL.  What do you all think?
o (MS): An alternative is to put one statement reflecting that security is 
out of the scope of the SpecGl, but this will probably be not beneficial a 
door will be open.
o (LR): If you read the scope it is limited to conformance topics, Security 
is not a conformance topic.
o (KD): As a conclusion, we all agree security out of the scope of the SpecGL.
o (LR): I will provide the response.
5.)TestGL topics (continued)
continue at #3 in [6]
more things about Test GL
TestGL issues list
draft about metadata and assertion
SOAP Assertions list [7]
XML Assertions list [8]
previous refs:  [6a], [6b], [6c], [6d]

(PC)Based of previous discussions, we believe that assertions are important 
but no necessary to be listed; they should be a priority 2 instead of 1.
(MS)We need to ensure traceability to either a test assertion or a requirement.
(PC)It has to be implicit of the purpose - identify the testable assertion. 
I need to put together a definition for assertion; a production in Xpath is 
a requirement that is a testable assertion. If the approach is not to list 
the assertions I need to reconstruct the GL's, we also need to discuss the 
minimum metadata and the alternative of associating the metadata with the 
test rather to assertions.
(SM)I think checkpoint 2.1 should be modified to relax the requirement for 
documenting or listing the assertions.
(PC): That will imply looking at CP2.2 the process of reviewing, marking 
what is testable, understanding the nature of the assertions has to be 
approach differently. Metadata should then be associated with the tests.
(DM): We should have a test catalog with metadata.
(PC):It can be formal or informal either way a test management system, the 
important data associated with test. Do we delete cp 2?
(DM): Every testcase must point into the specification to the least 
granular number section. Priority one should be that you should always 
point into the spec.
(MS):I still believe that the assertion must be identified and documented, 
but there are conditions were it is not necessary.
(PC): The more typical way is to developed the assertion or purpose and 
point to a section in the specification. Is not formally to list the 
assertion this is the commonality.
(MS): That is a separate issue. We should mandate assertions.
(PC): The only test suite that will pass this GL is SOAP.
(MS): The criterion is not what happened in the past.
(LR): The Accessibility Guidelines are in the same format as the assertions 
in SOAP, I will like to be able to say that I can point directly to that 
section in the specification.
(DM): SOAP is basically extracting sentences.
(MS):I think we should not be too specific. I will like to see a list, but 
only an attempt to document will be good.
(PC): We need a liberal definition. The idea is not to add extra work; if 
we have a normative statement in the specification that is testable we 
should be able to point to it.
(PC): I will remove the metadata section and combine in GL3. Is there 
something else?
(LR): We should add a statement on assertions being atomic.
(PC):Ok, GL2 will be on Identification and level of granularity and GL3 on 
test material management process. I will make the cataloging the heat of 
this. Lynne, can you provide with definitions of atomic? What kind of 
metadata need to me associated? what is the minimum set of metadata?, also 
what king of metadata is priority 2?
(DM): Sometimes you have one test case that goes across specifications.
(PC)This is in order to understand or classify the assertion. It might be 
necessary to address these circumstances in the assertion CP.
(DM):Identify assertion or combination of assertions.
(PC): That sounds reasonable. What about expected result? There are two 
classes of metadata; information related to the assertion and information 
on test suite management.
(DM) There is also the test review status, which is important for test 
management. Also a proper management system should be able to identify 
which range of versions of the specification it relates to. This is 
essential to represent compatibility across versions and desirable to avoid 
creating
an entire new suite for each version.

6.) Adjourn







Sandra I. Martinez
National Institute of Standards and Technology
100 Bureau Drive, Stop 8970,
Gaithersburg, Md. 20899

(301) 975-3579
sandra.martinez@nist.gov

Received on Monday, 22 September 2003 12:08:53 UTC