Re: Draft Minutes - QA Working Group Teleconference - 04-April-2005

Thanks for the detailed notes, Richard. I've interspersed some comments, 
responses, and questions below.


Kennedy, Richard T wrote:

>Summary of New Action Items:
>AI-20050404-01  (LH) Email QAWG to establish consensus for Dublin F2F dates                       2005-04-04
>  
>
I think this AI was for me rather than Lofton. However, since he has 
sent the message, let's leave it as is.

>AI-20050404-02  (PC) Publish "Test Development FAQ" to W3C QAWG web site                          2005-04-04
>  
>
Done

>AI-20050404-03  (PC) Remove "Working Draft" language from Test Development FAQ (TDFAQ)            2005-04-11
>  
>
Done

>AI-20050404-04  (PC) Define "living document" in TDFAQ Introduction                               2005-04-11
>  
>
Done

>AI-20050404-05  (DH) Define versioning for TDFAQ                                                  2005-04-11
>AI-20050404-06  (PC) Change all "see here" hyperlink descriptions in TDFAQ                        2005-04-11
>  
>
Done

>AI-20050404-07  (PC) Add language on starting test development early to TDFAQ Question 2          2005-04-11
>  
>
I don't think this AI is necessary - during the discussion I mentioned 
that I had already done this.

>AI-20050404-08  (PC) Update OWL reference in TDFAQ Question 2                                     2005-04-11
>  
>
I don't remember what this was about. (There was a separate discussion 
about the need to incorporate an additional OWL reference about their 
test review process, but I've already taken care of that.)

>AI-20050404-09  (PC) Add language on using other Working Group tests to TDFAQ Question 3          2005-04-11
>AI-20050404-10  (DH) Develop separate question for using other Working Group's tests              2005-06-06
>  
>
We originally suggested adding language to Q3, and then changed our 
minds and decided to create a separate question. So - please delete AI-09

>AI-20050404-11  (DH) Contact Voice Browser WG about the Test Case Tracking System                 2005-04-11
>AI-20050404-12  (PC) Define "results" in TDFAQ Question 6                                         2005-04-11
>AI-20050404-13  (PC) Add language on referencing other files to TDFAQ Question 6                  2005-04-11
>AI-20050404-14  (DH) Develop separate question discussing reference files                         2005-06-06
>  
>
I don't remember us suggesting that this should be a separate question. 
However, if we did please convert the AI to "PC to open a Bugzilla issue 
reminding us to expand on this issue in a future rev of the document"

>AI-20050404-15  (LR/DD) Provide examples of legal problems for TDFAQ Question 7                   2005-04-11
>AI-20050404-16  (PC) Add language on the importance of legal issues to TDFAQ Question 7           2005-04-11
>AI-20050404-17  (PC) Add language on separating test development & execution to TDFAQ Question 8  2005-04-11
>AI-20050404-18  (PC) TDFAQ Question 9 missing                                                     2005-04-11
>  
>
Done

>AI-20050404-19  (LR) Provide XML Core Test Suites examples for TDFAQ Question 11                  2005-04-11
>AI-20050404-20  (DD) Rewrite 1st sentence of TDFAQ Question 12                                    2005-04-11
>AI-20050404-21  (PC) Review remaining TDFAQ Question & provide examples                           2005-04-11
>  
>
This should be: "request examples if appropriate" rather than "provide 
examples"

>Agenda: http://lists.w3.org/Archives/Public/www-qa-wg/2005Apr/0003.html
>Previous Telcon Minutes: http://www.w3.org/2005/03/21-qa-minutes.html
>
>Minutes:
>
>Next Meeting: April 11  - The topic is the SpecGL.
>RK sends regrets for April 11.
>
>
>
>Dublin F2F
>Prior date consensus was August 9-11 (Tuesday-Thursday). LH suggested August 8-10 (Monday-Wednesday) because those coming from North America, who do not have a nice corporate/government travel package, often need a Saturday night stay to get reasonable fares. Plus, August is still high travel season and he is finding that traveling for a Monday-Wednesday meeting improves the choices and fares available. LH will email the QAWG for consensus on the F2F dates. Exact meeting location and recommended hotels were not discussed.
>
>
>
>Test Development FAQ discussion (Lead by PC)
>Latest draft: http://lists.w3.org/Archives/Public/www-qa-wg/2005Apr/att-0002/TestFAQ-20050403.html
>
>We are getting very close to publication especially since this is a living document. The FAQ's format will be that of a Note on the QAWG web page. PC will publish the current version on the QAWG web site today.
>
>Labeling the document as a "working draft" implies it is a Specification rather than a Note. PC will remove this text.
>
>
>Introduction:
>There was some general discussion on the audience for this note; it is W3C or outside persons?
>
>PC will add additional language defining a "living document."
>
>DM expressed concern on how others will link to this document. How will the different versions be identified? DH will discuss with DM and pass on recommended method to PC.
>
>
>Question 1: What kinds of testing is important in the W3C?
>PC will add descriptive text to all hyperlinks with text like "see here" to prevent accessibility issues.
>
>
>Question 2: When should test development start?
>PC will add language stating the importance of starting test development early, especially before the Specification is frozen. The OWL reference also needs to be updated.
>  
>
See AI annotations above.

>
>Question 3: Who will develop the tests?
>The desirability in including tests developed by other Working Groups was discussed. New Specification are increasing modular making their reuse easier. PC will add a few sentences discussing this. DH will look at making this a separate question after the first publication of this document.
>  
>
See AI annotations above. Please add another AI for PC to open a 
Bugzilla issue reminding us that we might want to expand on the 'test 
re-use' question in a separate document.

>Using XForms as an example may be too cutting edge.
>
>The Voice Browser Working Group may have a test case tracking system. We need to track this system down.
>
>
>Question 4: How do we decide what tests to develop?
>PC still needs additional example for this question. He mentioned that at a recent meeting at Sun, the XQuery Working Group mentioned their separate test task force. However, they have not made any solicitation for tests from outside of their Working Group.
>
>
>Question 5: How many tests are enough?
>In the current examples nobody is publishing numbers or measurements. PC mentioned that test suites need to be versioned and associated with the corresponding version of the Specification. DH stated that coverage numbers may be too technical for inclusion in this document. PC will add some language illustrating some techniques for %%@ HELP. I need a definition of what was discussed. @%%
>  
>
"illustrating some techniques for coverage measurement" will be 
sufficient for these minutes. However, we also need an AI (for me) to 
open and Issue in Bugzilla suggesting that we expand the discussion of 
coverage metrics in a separate document.

>TB supplied some examples: 
>http://www.w3.org/WAI/GL/WCAG20/tests/
>HTML WCAG2.0 Test Suite (sorted by Guideline)  - gives some indication of test coverage by Guideline and level within Guideline
>
>Example for "reference implementation" for testing: http://www.w3.org/Math/testsuite/ MathML2.0 Test Suite - gives "reference" renderings to compare against the renderings under test for each test in test suite.
>
>
>Question 6: How should tests report their results?
>DH mentioned that the five test-results states are very useful. PC requested additional examples.
>
>The term "results" can mean the output of administering a test or the pass/fail outcome of a test. PC will define this in the document.
>
>PC will add a sentence describing reference files and what actual files are compared. DH will add this as a separate question after publication.
>  
>
See AI annotations above

>
>Question 7: Do I really have to worry about all that legal stuff? LR and DD will provide some examples of legal problems Working Groups have had with their Specifications. DH will all language to the question stressing the importance of legal issues.
>
>
>Question 8: How should I package and publish my tests?
>PC can not find anyone currently doing this. The SVG Working Group package does not have any documentation in its zipped test suites. The answer should stress that there are different audiences. A document on how to execute tests should be kept separate from a documents describing how to develop tests. PC will add appropriate language.
>
>
>Question 9 (Missing)
>PC to renumber document questions.
>
>
>Question 10: Should I automate test execution?
>More examples are needed. The SVG Working Group might have something useful as they provide harnesses in their test suites as does the DOM Working Group. When XQuery's work is made public, they will also have one.
>
>
>Question 11: Once I publish my tests, I'm done, right?
>More examples are needed and the text in red will be stricken. LR will provide a link to reflect the XML Core Test Suites that have been both augmented and corrected.
>
>
>Question 12: How should I handle bugs in my test suite?
>DD will rewrite the first sentence.
>
>
>At this point our telcon ended. PC will review the three remaining questions and look for examples. The overall consensus is that we are very close to publishing this document.
>
>
>  
>

Received on Tuesday, 5 April 2005 22:21:01 UTC