- From: Kennedy, Richard T <richard.t.kennedy@boeing.com>
- Date: Tue, 5 Apr 2005 14:14:14 -0700
- To: <www-qa-wg@w3.org>
NOTE: (1) Most of the AI completion dates are a guess as they were not discussed. Please correct as needed.
(2) Please have your comments to me before Friday, April 8. Otherwise I can't publish the final minutes until the week of April 17.
QA Working Group Teleconference
Wednesday, 04-April-2005
--
Scribe: Richard Kennedy
Attendees:
(TB) Tim Boland (NIST)
(PC) Patrick Curran (Sun Microsystems)
(DD) Dimitris Dimitriadis (Ontologicon)
(DH) Dominique Hazaël-Massieux (W3C)
(LH) Lofton Henderson (CGMO)
(RK) Richard Kennedy (Boeing)
(LR) Lynne Rosenthal (NIST)
(DM) Dave Marston (Guest)
Regrets:
(KD) Karl Dubost (W3C, Chair)
(MS) Mark Skall (NIST)
Absent:
None
Summary of New Action Items:
AI-20050404-01 (LH) Email QAWG to establish consensus for Dublin F2F dates 2005-04-04
AI-20050404-02 (PC) Publish "Test Development FAQ" to W3C QAWG web site 2005-04-04
AI-20050404-03 (PC) Remove "Working Draft" language from Test Development FAQ (TDFAQ) 2005-04-11
AI-20050404-04 (PC) Define "living document" in TDFAQ Introduction 2005-04-11
AI-20050404-05 (DH) Define versioning for TDFAQ 2005-04-11
AI-20050404-06 (PC) Change all "see here" hyperlink descriptions in TDFAQ 2005-04-11
AI-20050404-07 (PC) Add language on starting test development early to TDFAQ Question 2 2005-04-11
AI-20050404-08 (PC) Update OWL reference in TDFAQ Question 2 2005-04-11
AI-20050404-09 (PC) Add language on using other Working Group tests to TDFAQ Question 3 2005-04-11
AI-20050404-10 (DH) Develop separate question for using other Working Group's tests 2005-06-06
AI-20050404-11 (DH) Contact Voice Browser WG about the Test Case Tracking System 2005-04-11
AI-20050404-12 (PC) Define "results" in TDFAQ Question 6 2005-04-11
AI-20050404-13 (PC) Add language on referencing other files to TDFAQ Question 6 2005-04-11
AI-20050404-14 (DH) Develop separate question discussing reference files 2005-06-06
AI-20050404-15 (LR/DD) Provide examples of legal problems for TDFAQ Question 7 2005-04-11
AI-20050404-16 (PC) Add language on the importance of legal issues to TDFAQ Question 7 2005-04-11
AI-20050404-17 (PC) Add language on separating test development & execution to TDFAQ Question 8 2005-04-11
AI-20050404-18 (PC) TDFAQ Question 9 missing 2005-04-11
AI-20050404-19 (LR) Provide XML Core Test Suites examples for TDFAQ Question 11 2005-04-11
AI-20050404-20 (DD) Rewrite 1st sentence of TDFAQ Question 12 2005-04-11
AI-20050404-21 (PC) Review remaining TDFAQ Question & provide examples 2005-04-11
Agenda: http://lists.w3.org/Archives/Public/www-qa-wg/2005Apr/0003.html
Previous Telcon Minutes: http://www.w3.org/2005/03/21-qa-minutes.html
Minutes:
Next Meeting: April 11 - The topic is the SpecGL.
RK sends regrets for April 11.
Dublin F2F
Prior date consensus was August 9-11 (Tuesday-Thursday). LH suggested August 8-10 (Monday-Wednesday) because those coming from North America, who do not have a nice corporate/government travel package, often need a Saturday night stay to get reasonable fares. Plus, August is still high travel season and he is finding that traveling for a Monday-Wednesday meeting improves the choices and fares available. LH will email the QAWG for consensus on the F2F dates. Exact meeting location and recommended hotels were not discussed.
Test Development FAQ discussion (Lead by PC)
Latest draft: http://lists.w3.org/Archives/Public/www-qa-wg/2005Apr/att-0002/TestFAQ-20050403.html
We are getting very close to publication especially since this is a living document. The FAQ's format will be that of a Note on the QAWG web page. PC will publish the current version on the QAWG web site today.
Labeling the document as a "working draft" implies it is a Specification rather than a Note. PC will remove this text.
Introduction:
There was some general discussion on the audience for this note; it is W3C or outside persons?
PC will add additional language defining a "living document."
DM expressed concern on how others will link to this document. How will the different versions be identified? DH will discuss with DM and pass on recommended method to PC.
Question 1: What kinds of testing is important in the W3C?
PC will add descriptive text to all hyperlinks with text like "see here" to prevent accessibility issues.
Question 2: When should test development start?
PC will add language stating the importance of starting test development early, especially before the Specification is frozen. The OWL reference also needs to be updated.
Question 3: Who will develop the tests?
The desirability in including tests developed by other Working Groups was discussed. New Specification are increasing modular making their reuse easier. PC will add a few sentences discussing this. DH will look at making this a separate question after the first publication of this document.
Using XForms as an example may be too cutting edge.
The Voice Browser Working Group may have a test case tracking system. We need to track this system down.
Question 4: How do we decide what tests to develop?
PC still needs additional example for this question. He mentioned that at a recent meeting at Sun, the XQuery Working Group mentioned their separate test task force. However, they have not made any solicitation for tests from outside of their Working Group.
Question 5: How many tests are enough?
In the current examples nobody is publishing numbers or measurements. PC mentioned that test suites need to be versioned and associated with the corresponding version of the Specification. DH stated that coverage numbers may be too technical for inclusion in this document. PC will add some language illustrating some techniques for %%@ HELP. I need a definition of what was discussed. @%%
TB supplied some examples:
http://www.w3.org/WAI/GL/WCAG20/tests/
HTML WCAG2.0 Test Suite (sorted by Guideline) - gives some indication of test coverage by Guideline and level within Guideline
Example for "reference implementation" for testing: http://www.w3.org/Math/testsuite/ MathML2.0 Test Suite - gives "reference" renderings to compare against the renderings under test for each test in test suite.
Question 6: How should tests report their results?
DH mentioned that the five test-results states are very useful. PC requested additional examples.
The term "results" can mean the output of administering a test or the pass/fail outcome of a test. PC will define this in the document.
PC will add a sentence describing reference files and what actual files are compared. DH will add this as a separate question after publication.
Question 7: Do I really have to worry about all that legal stuff? LR and DD will provide some examples of legal problems Working Groups have had with their Specifications. DH will all language to the question stressing the importance of legal issues.
Question 8: How should I package and publish my tests?
PC can not find anyone currently doing this. The SVG Working Group package does not have any documentation in its zipped test suites. The answer should stress that there are different audiences. A document on how to execute tests should be kept separate from a documents describing how to develop tests. PC will add appropriate language.
Question 9 (Missing)
PC to renumber document questions.
Question 10: Should I automate test execution?
More examples are needed. The SVG Working Group might have something useful as they provide harnesses in their test suites as does the DOM Working Group. When XQuery's work is made public, they will also have one.
Question 11: Once I publish my tests, I'm done, right?
More examples are needed and the text in red will be stricken. LR will provide a link to reflect the XML Core Test Suites that have been both augmented and corrected.
Question 12: How should I handle bugs in my test suite?
DD will rewrite the first sentence.
At this point our telcon ended. PC will review the three remaining questions and look for examples. The overall consensus is that we are very close to publishing this document.
Received on Tuesday, 5 April 2005 21:14:20 UTC