draft minutes of QA F2F meeting in Boulder, 22 Oct AM

Here follows the draft minutes of our meeting this morning; please send
your corrections along:


SpecGL CR Plan
For SpecGL, Karl will send the CR request today, which would lead us to
a call next week. That would leave us around 2 weeks before the
publicaton moratorium. The document itself is close to be ready (only a
few fixes need to be applied after the latest resolutions). But the
implementation plan might need more work, if we want to show that SpecGL
has already been used to do review and to show that some specifications
already implement a significant coverage of the CP of SpecGL, so that we
can show that specGL is really anchored in the experience of other specs
development.
AI-20031022-1 Dom to prepare the specGL implementation plan with
data/historical background by 2003-10-29

According to Karl, some groups are more interested in SpecGL than OpsGL
(e.g. I18N) and it might be easier to get commitments from other groups
to apply them. Groups to contact:
- WCAG
- I18N
- SVG?
AI-20031022-2 Lofton to ask Chris or Dean if they could be interested in
applying specGL by 2003-10-29
- CSS?
- XPath2.0?

Ops CR Plan
We have 6 WGs with which we are supposed to work. Current status:
- DOM -> no
- I18N -> why not? but no follow-up
- OWL -> yes, but with a second candidate in the WG
- SVG -> added to the agenda, but no news; taken on by Lofton
- WCAG -> why not? but no news; taken on by Olivier
- WSD -> on the agenda, but no reply; taken on by Dom
AI-20031022-3 Karl to recontact all his potential groups to get an
update by 2003-10-31
AI-20031022-4 Karl, Dom, Olivier, Lofton to report on their efforts with
the above mentionned WG by 2003-11-28
We'll need to make a report to W3C management by end of December and see
what's the status at that point.

Short package of selling points
Focus on outreach will be put on SpecGL first, and OpsGL only
thereafter, since SpecGL seems to be easier to adopt. The idea would be
to reuse some of the work done for the QA Outreach kit during last TP:
http://www.w3.org/2003/Talks/qa-outreach/
but with a more focussed approach on selling points. Ideas:
- to show that it may ask more work, but allows to get quicker to a
final stage; but no clear experience to prove that
- audience is not necessarily hostile to QA practices; need only to be
convinced that Spec and OpsGL are good tools, and that they provide
useful input
- GL as checklists for things not to forget; to make it easier to create
some of the work with templates, tools
- QA is already partly incorporated in W3C process (QA Team on the call,
TS at CR), we can re-use that argument
- allows to reduce risk  at end of spec development; less time to revise
(e.g. SVG CRs)
- getting testimonials of people showing how GL in Ops and Spec could
have made them save time
AI-20031022-5 Lynne to have a first cut at the selling points based on
the brainstorming output by 2003-10-31


TestGL discussion
http://www.w3.org/QA/WG/2003/10/TestGL-20031020.html
Patrick had a bit of difficulty integrating the comments received during
the past telecon, because they were not specific enough.
***
Regarding introduction: should be on the same structure as introduction
of other GLs.
CoP: test materials
Audience: WG, tests developpers and users. Primary audience: people
developping tests; secondary audience: people using tests, spec
developpers.
***
Concepts section: "function testing" added; the goal is to cover
performance, usability, implementation-dependant features that are not
covered by the spec (which is conformance testing) -> should be renamed
as "other types of testing" (performance doesn't belong to function
testing).
Discussion of differences between interoperability/conformance.
Interoperability is possible without conformance, but it needs prior
agreement, and it doesn't scale in numbers of implementations, or with
time.
Need of detailing how to build interop testing? Alex points that
conformance testing and interop testing are just specfic cases of
functions testing, as they are just comparing expected results described
in a document. This will be explicited in the concepts section, but the
focus of the document will be kept on conformance testing.
---
Tests dev strategies: tests has 3 possible uses:
- help the user detect if the implementation is conformant or not
- help the implementation developer improve his product
- help the specification writer improve his specification
The section should stress that we don't necessarily require a
"waterfall" process, and that the requirements described in the GL can
be applied on a recursive basis; we should actually encourage this type
of strategy (without requiring it) in this section.
-> good to detail in use-cases:
* a testing lab/certification authority needs to test products; the spec
and the products already exists, only needs to check if the products
conforms
* implementations are begun before spec is finished; needs tests to
check if their implementations confoms
* WG needs feedback on its specification, and uses test cases as a way
to get this feedback
* WG uses test cases as a way to explore new features (eg in OWL, SVG,
CSS)
* comparisons of actual state of implementations independantly of
conformance [example of interop testing]
---
Assertions
Overlap with specGL?
the WG reaffirms the need to duplicate the TA CP in this TestGL, since
some specs don't provide test assertions. In specGL, TA are considered
as output, while in TestGL, they are input.
Reviewing the TA def in specGL, it appears we also want to move some of
the verbiage of SpecGL GL10 to the definition in specGL glossary and
QA-Glossary. Discussion of what "complete" in this definition means:
self-contained, comprehensive, testable?
AI-20031022-6 Dom to update the glossaries definitions of TA in SpecGL
to include the 2nd sentence of GL10 TA def by 2003-10-29
The assertions section of TestGL should point to the def, address why
they are important, how they can be extracted/derived automatically or
not, address what makes a good/useful TA.
Needs to emphasize again that there might be a feedback loop between the
Test assertions extraction and the spec development.

Dom
-- 
Dominique Hazaël-Massieux - http://www.w3.org/People/Dom/
W3C/ERCIM
mailto:dom@w3.org