MINUTES: W3C User Agent Special Telecon on Test Suites on July 18th, 2002

HB: Harvey Bingham
CK: Colin Koteles
MM: Matt May
DP: David Poehlman
DD: Dimiti Dimitriadis
LH: Lofton Henderson
JG: Jon Gunderson
TL: Tim Lacy
IJ: Ian Jacobs
EH: Eric Hansen


Introduction

JG: reviews the test suite documents

LH: It looks good to me

DD: What kind of lines for the DTD
Have you tried to capture specific browser behavior

JG: We have not used any propriety behavior of a user agent

JG: recap

LH: I thought the thrust of DD question, does the framework require a 
specific browser behavior

JG: No

DD: Does the presupposed any html behavior or subset of HTML

CK: We use the HTML DTD for the code fragments

LH: We have an XML DTD in DOM TS, that fully implements the DOM.  We use 
the style sheet produces an XML schema.  What kind of compliance of the 
browser do you expect.
What we have done, we test things that are never implemented.  Some things 
are rarely implemented.  By using the XML versino we have come into 
situtations where the tests becomes more complex than it needs to be.

HB: Tunnelized implementors.  The shorter versino could be a java script 
instead of XML.

LH: The problem with using the XML version for the DTD, we use call to the 
XSL engines that are the same as DOM calls.  So you need a 100% browser 
comaptibility of the browser.  You have a radio button that adds a button 
to an ordered list.  If the browser doesn't comply then you can't test this 
condition.

IJ: Dropping off

EH: Dropping off

HB: Earl work?

LH: We are aware of Earl.  We saw a demo of a tool.

HB: A lot of good work.  But no one seems to be applying it.

LH: The working groups developing test suites are using there own formats 
right now.

DD: Leaves

LH: We will be looking at an application of EARL

LH: Main barrier of earl needs front end and report generating tools.  This 
is something that we hope we can do in a couple months to producing 
tools.  Working groups are doing the tools themselves right now.  We will 
atart tools develop in a few months, will look at EARL as a reporting language.

JG: I work with student groups, let me know if that will help

LH: Starting to form the tools group

JG: What about non-machine testable questions

LH: Others are more knowledge, I am more familar with automation like DOM 
and graghics

HB: are you saying that it can always be automated

LH: In DOM they are generating the test suite from the 
specification.  These syntaxes are all computation based.  The automation 
in something like SVG is not as easy to automated.  SVG is all about 
semantics, how to draw the picture.  UAAG seems more like SVG, than DOM 
level 1.  Just thinking outloud, there are automatied test features.

JG: I like the idea of generating the test suite from the from a shared source.

HB: Any idea about reporting results

LH: We have requirements for that.  One requirement is tomeasure the extent 
of implementation.  We are still wokring on this requirement. In about a 
month we will be developing and pointing to techniques.

JG: Normative exceptions

LH: Ian and I had a dialogue a few months ago.  I disagreed with his view, 
but reading the 8 February requirement document for test suties.  Sounds 
like his thinking are more alike, in that the exceptions are not really 
requirements but are just part of the pass fail criteria.

JG: talked about conformance model

JG: Test suite navigation

LH: The QA editors that it is part of the test suite hareness, my own 
thoughts on navigation are that basically that you ought to have order 
through the test suite: next, last, and random access (TOC).

HB: What is a hareness?

LH: Factilitates organization of the tests and the results.

DP: Leaves

HB: Joy to be in the presences of LH

LH: A appologize for the lack of QA participation

JG: Would you please consider using the TS for possible TS techniques

LH: A good timing for QA for another meeting dependent on an 19 August 2002 
publishing date.  Maybe after this date would be a good time. About a month.

Received on Thursday, 18 July 2002 15:04:24 UTC