Final Minutes: QA Working Group Teleconference 2003-02-24

QA Working Group Teleconference
Monday, 24 February -2003
--
Scribe: Lynne Rosenthal

Attendees:
(PC) Patrick Curran (Sun Microsystems)
(KD) Karl Dubost (W3C, WG co-chair)
(KG) Kirill Gavrylyuk (Microsoft)
(DH) Dominique Hazaël-Massieux (W3C) -
(LH) Lofton Henderson (CGMO - WG co-chair)
(SM) Sandra Martinez (NIST)
(LR) Lynne Rosenthal (NIST - IG co-chair)

Visitor
(JR) Joseph Reagle (W3C)

Regrets:
(dd) Dimitris Dimitriadis (Ontologicon)
  (MS) Mark Skall (NIST)
(AT) Andrew Thackrah (Open Group)


Absent:
(PF) Peter Fawcett (RealNetworks)

Summary of New Action Items:
AI-2003-0224-1: JR to summarize discussion, Feb 25

Agenda: http://lists.w3.org/Archives/Public/www-qa-wg/2003Feb/0115.html
Previous Telecon Minutes:
http://lists.w3.org/Archives/Public/www-qa-wg/2003Feb/0112.html


1.) Roll call 11am (ET), membership

2) Licenses for Test Materials.
JR will summarize the discussion and conclusions, let WG review and then, 
present to W3C management and legal. Teleconference facilities available in 
Boston if we need to continue the discussion.

Discussion follows [1a]
1.1 Nature of test materials (terminology)
1.1 test cases (documents)
1.2 test software/harness (software)
1.3 granularity
The agreed upon terminology was test cases, test software and test 
documentation.
Often it is necessary to supplement the test suite provided by the WG in 
order to run it. A WG may specify the bindings and abstract 
representation.  The user would need to do the binding  augment the test 
cases with the binding, harness, etc.  This is a useful distinction.

KG indicated that a collection of test cases may not be used as a whole, 
but as a collection. Test cases can be used, subsumed into other sets.  Is 
this something to that should be captured in the license? JR stated that it 
the test cases and harness can be treated in separate licenses.  This is an 
option to consider.

#2. Common license requirements
2.1 warranties/liabilities.  KG asked about indemnifications and 
limitations on distribution. His concern is mostly with test software.  JR 
states that there are two views: distribute materials under a license and 
use materials under a license. KG is concerned about being able to take the 
software and incorporate it into his own company’s software and distribute 
it.  With redistribution, limit the problems of warranties/liabilities, be 
protected from making a mistake. JR  since W3C is not willing to indemnify 
the use of these materials, then a company would be at risk for using these 
materials.  Also, not sure that a W3C change would help this 
situation.  W3C wants companies to distribute, W3C restrictions to 
distribution may have unintended consequences, e.g., interfere with 
mirroring.   PC thinks that changing redistribution may not be the fix, 
rather using some standard language to handle the liability question may work.

2.2 scope of use. Concerned with limiting the damages from potential 
misuses. Provide some protection and represent the intentional use of test 
materials. Competitors can use submitted tests in a way that wasn’t 
intended (and can’t control) that could cause damage. Doesn’t this relate 
back to warranties/liabilities?  Isn’t this more of a competitive issue. 
Specifically, you are happy to donate tests but not want competitors to use 
against you.  JR: any examples of this? KG: could be a test case that can 
bring down a web service.  This hasn’t been something addressed by W3C and 
it isn’t clear how to enforce this. Some licenses out there go beyond 
copyright.  W3C is not trying to create end user shrink wrap that is 
restricting other things.  Approach this with caution. LH there doesn’t 
seem to be consensus about this issue. PC: the competitive aspect can be 
handled in other ways, with respect to malicious use, can leave this up to 
the law.  JR: if you change the tests so that you can pass them and claim 
conformance, this would get resolved by the market (egg on your face) and 
violate copyright. This could also be handled by trademark or other 
strategies.

2.3 Redistribution.
KG: redistribution: limit the problems of warrenties/liabilities in that 
you might not want your customers to have these things. Concern for the 
test environment (code/harness)
PC: Test suites downloaded from W3C are not whole, still need to make 
modifications and/or tailor them to make them useful in the real world.  A 
company may want to pass these changes to their customers.  Prohibiting 
redistribution would make this difficult.  For example, a company may want 
to make additional modifications to the test suite or incorporate them into 
their own test harness or even substitute a test harness. KG does the same 
- not use harnesses of public test suite, use their own.  SVG distributes 3 
different harness, but we all know that nothing is all complete.  CR: But 
put the distinction on test cases, as opposed to all the documentation, 
harness, etc that needs to be present.  There is a spectrum in W3 of what 
is produced: those that are near complete and those that need additional 
pieces.  Providing a standardized way to describe the test cases, may 
enable the development of a generic test harness. Collection of test cases 
and metadata could be assembled by those using it (or distributors).  JR: 
this sounds like a useful future resource. Until assured, we don’t want to 
limit redistribution. Will still want to let people take something from W3C 
and map it to their processes and then redistribute. JR: this will be a new 
issue in my summary, i.e., potential harm (liability) when give test suites 
to your customer.  This is not a first class issue in of itself. LH: we 
have drifted into 2.4 modification/derivation work.  Hard to write 
something that fits all scenarios. KG: document license lets you copy pages 
as long as you don’t redistribute.  JR: This is called fair-use.  Can get 
special permissions for translations, etc.  FAQ document explains these.

Discussion about modifying test suites to make them complete.  DOM provides 
a core set of test cases that need additional stuff added to them.  SVG 
doesn’t.  Maybe there needs to be selective restrictions depending on what 
component of the test suite you are dealing with.  Do we want to restrict a 
company from combining test cases from various W3C test suites or 
incorporate into a broader company test suite.  What if some of the W3C 
tests are wrong and in the short term, a company corrects them.  You 
shouldn’t exclude or change test but annotation them to indicate that they 
are not correct and in process to correct.  Some harnesses have the ability 
to exclude those things that they think are buggy.  As long as you don’t 
claim conformance to the W3C tests, then this is not a problem.  JR: The 
easiest way to deal with this is the Software license that permits 
adaptation (creation of derivative work) as long as you do not make a claim 
of conformance to W3C test suite. KG: if our concerns about liability were 
addressed we wouldn't need to restrict modification/derivation.  PC: is it 
possible to create derived work, publish and sell it?  JR: no problem with 
members putting it into member products and selling, but must put in 
software license disclaimers and acknowledgements. PC:  we are under 
pressure to create derivative test suites but we fear this can lead to a 
dilution of the "one true" test suite.

2.5 Link
PC: Wants an explicit grant to invoke and apply (e.g., to change 
interfaces), but not the right to change test cases.  JR: document license 
grants the right to make a copy, make adaptations, deriviations, it is an 
issue to add public performance and public display.  KG indicated that the 
Document license might be appropriate to the document test cases.

Discussion to continue at F2F
   JR: to write summary of discussion  wed.

Ajourn 12:30 (ET)

[1a] http://lists.w3.org/Archives/Public/www-qa-wg/2003Feb/0111.html
[1b] http://lists.w3.org/Archives/Public/www-qa-wg/2003Feb/0042.html

[2a] http://lists.w3.org/Archives/Public/www-qa-wg/2003Feb/0092.html
[2b] http://lists.w3.org/Archives/Public/www-qa-wg/2003Feb/0103.html
[3a] http://lists.w3.org/Archives/Public/www-qa-wg/2002Nov/0044.html
[3b] http://lists.w3.org/Archives/Public/www-qa-wg/2003Jan/0027.html
[3c] http://lists.w3.org/Archives/Public/www-qa-wg/2003Jan/0041.html
[3d] http://lists.w3.org/Archives/Public/www-qa-wg/2003Feb/0010.html

[4] http://lists.w3.org/Archives/Public/www-qa-wg/2003Feb/0077.html
[5a] http://lists.w3.org/Archives/Public/www-qa-wg/2003Feb/0089.html
[5b] http://lists.w3.org/Archives/Public/www-qa-wg/2003Feb/0104.html

Received on Tuesday, 4 March 2003 07:16:48 UTC