Testing call this Monday

Folks,

It looks like everyone can be on the call this Monday at 9am ET. Let's
meet on #testing as usual.

agenda+ Closing on the requirements
agenda+ Testing Interest Group charter (Plh and Mike)
agenda+ Mozilla testing framework (Francois)
agenda+ Starting point for the framework
agenda+ Documentation for Groups
agenda+ Organizing w3c-test.org

Anything else?

More info:

On the testing interest group: this idea came out of Jeff to give a
"formal" status within W3C, raise the profile of the project, and allow
others to feel they're welcome to contribute to the project. Mike is
working on a draft charter for that. As usual, the trick is to restrict
the scope of the Group so that folks don't think we're going to test
everything everywhere. Maybe something along the lines of "HTML5 Testing
Interest Group" but we'll still need to differentiate it from the HTML5
test suite task force. The goal of the interest group would not be to
produce tests in any case. By the way, we would need to find a chair (or
co-chairs?) for the IG and it would be nice if we avoid having me on the
critical path. On the other end, I have a vested interest in ensuring
that the testing project is successful so I'm ok to be the default
candidate to chair.

On the starting point for the framework: I wonder how long we should
take to close the loop on that one. Either we take the current HTML
framework and continue it, or we use an other one for the starting
point. In any case, I would hate for us starting from scratch and
reinventing the wheel. James mentioned that he has one that can run the
HTML5 parsers already so maybe we could explore that path as well? Also, we'll
need to figure if the framework to be used on desktop browsers will be
the same one as the framework to be used on mobile browsers. Again, if
we can avoid duplicating efforts, that would be better. It may well be
that we start on a common framework and branch it later on.

On the documentation for Groups: this came up from a discussion with Art
Barstow related to organizing the test suites in the WebApps group. A Group
interested in testing their spec will look for documentation on, at
least:
 - framework (which should they use?, how to write a good test)
 - process (submitting tests, reviewing)
 - test management (mercurial, test servers, license, organizing the
directories, annotations, etc.)
 - test result management (collecting, publishing, etc.)
We need to help them by providing guidelines, examples of process, etc.
I don't expect that one size will fit all so it's fine to have
alternatives, including for the test framework and process btw. Art has
produced various documents for the WebApps group recently so may
be we could reuse those as a starting point.

On organizing w3c-test.org: the server is likely to become a mess in the
next 6 months due to various groups switching to it and publishing their
tests in various directories. I don't want to impose rules on the groups
but I do wish there was an easy way to discover test suites on the
server. One idea: we recommend that each test suite has an HTML page (or
a set of HTML pages?) that links to the test files. We could impose a
little bit of markup in it such as "spec tested" and "group" at the minimum.
Those pages would be listed in some master file somewhere, like
"http://w3c-test.org/testsuites.txt". From that information, we could
generate a nice master page for "http://w3c-test.org/" that would provide
links to the various test suites available on the server.

As a side note, I recommend that folks play a bit with testharness.js.
The more feedback we can get back to James, the sooner we'll get it
stable.

Philippe

Received on Thursday, 31 March 2011 05:41:15 UTC