[minutes] Testing call on Monday 9 May 2011

Ref: <http://www.w3.org/2011/05/09-testing-minutes.html>


Best,
   Shadi


                          W3C Testing Framework

09 May 2011

    See also: [2]IRC log

       [2] http://www.w3.org/2011/05/09-testing-irc

Attendees

    Present
           Plh, Shadi, Francois, MikeSmith, Wilhelm

    Regrets
    Chair
           Philippe

    Scribe
           shadi

Contents

      * [3]Topics
          1. [4]Testing IG
          2. [5]CSS Testing
          3. [6]Framework
      * [7]Summary of Action Items
      _________________________________________________________

Testing IG

    <MikeSmith> [8]http://www.w3.org/2011/05/testing-ig-charter.html

       [8] http://www.w3.org/2011/05/testing-ig-charter.html

    MS: did several changes since we last talked
    ... talked with Philippe about Task Forces
    ... worked well in several settings

    <MikeSmith> example: [9]http://www.w3.org/2004/04/wcag-charter

       [9] http://www.w3.org/2004/04/wcag-charter

    shadi: where would you envision Task Forces? the core scope seems
    pretty broad anyway

    <Zakim> MikeSmith, you wanted to say that there are many parts of
    the shared testing infrastructure that are common to testing of
    different classes of apps

    mikeS: did not want to limit the authority of creating TFs
    ... to create reusable infrastructure
    ... for instance like a commenting funcationality that is not
    limited to a particular class of applications
    ... distguishing that from the test runner for the browser, which is
    specific to particular types of applications

    shadi: think that makes sense but may need to have better
    understanding of the individual components, interfaces, and
    terminology for these

    <francois> +1 to "test runner"

    philippe: next would be to send the IG Charter to the AC, so that I
    can talk with the AC folks next week
    ... try to get review period as soon as possible

    [10]http://www.w3.org/WAI/CG/

      [10] http://www.w3.org/WAI/CG/

CSS Testing

    philippe: previously looked at existing work
    ... have a "test runner" for HTML5
    ... also one from Mozilla that seems to be more complete
    ... for instance, it can display two documents for comparison, which
    the HTML5 one cannot do
    ... also know there are other "runners" elsewhere
    ... Dom developed one for mobileWeb and there is another one for CSS
    somewhere
    ... so question is do we start by building on top of an existing
    runner, and if so which
    ... have a call on that today at noon ET
    ... trade-off between metadata and how they will be added
    ... would be good to learn how the CSS folks managed 9k+ tests
    ... to have ready documentation next time someone wants to
    contribute tests

    francois: already reviewed some of the CSS testing work
    ... some documents we should be familiar with before the call

    philippe: purely informational call

    <plh-home> [11]http://wiki.csswg.org/test

      [11] http://wiki.csswg.org/test

    wilhelm: should also look into how the different browser vendors use
    the tests
    ... the importing is really painful
    ... partially manual and very messy
    ... would like to know how the other browser vendors handle this
    ... and the IG could facilitate this process
    ... also good to look at automation

    philippe: automation is an important aspect
    ... not easy to automate browsers across platforms
    ... but could provide the necessary data
    ... maybe could automate such tests in the future but not in the
    near future
    ... also need to educate people on how to write automatable testsd

    s/testd/tests

    scribe: as to facilitating the use of test suites for browser
    vendors
    ... also a goal but not yet clear how
    ... if we do our job well, we can harmonize the techniques

    shadi: all sounds like metadata is necessary to provide the
    necessary interface

    philippe: metadata is bigger than conventions
    ... different groups are more specific about their assertions
    ... even create their specs accordingly
    ... others are much less involved
    ... we have w3c-test.org but difficult to get an overview

Framework

    philippe: now called "test runner"
    ... need first version by summer
    ... question is which "test runner" is to be first

    shadi: thought this group needs to develop more than the "test
    runner" alone but also the infrastructure around it
    ... like test repository, test metadata, and other parts of what we
    talked about

    philippe: yes, the group needs to develop the other pieces
    ... but need to have the runner ready by mid-summer
    ... need to develop the pieces along the way

    shadi: need a description of the different components and interfaces

    philippe: yes, groups should be able to create their own test
    runners
    ... would like to avoid forking as much as possible
    ... but may be unavoidable in specific situations

    francois: we sometimes have to bundle test cases different in some
    cases
    ... does not mean we need cannot have a common repository
    ... just a matter of repackaging

    philippe: need to develop those diagrams

Summary of Action Items

    [End of minutes]


-- 
Shadi Abou-Zahra - http://www.w3.org/People/shadi/ |
   WAI International Program Office Activity Lead   |
  W3C Evaluation & Repair Tools Working Group Chair |

Received on Monday, 9 May 2011 14:07:00 UTC