Re: RfC: WebApps Testing Process

On Fri, Apr 1, 2011 at 1:26 PM, Arthur Barstow <art.barstow@nokia.com> wrote:
> I'm not sure we need to explicitly designate test suite maintainers.

I'd be okay with not having specific maintainers, but then we need to
figure out some good process for what to do if someone finds a test
bug.  With a spec, you can file a bug and the editor gets CCd.  The
editor is responsible to fix it, and also has the ability to fix it.
If no one is specifically designated as test maintainer for each spec,
then who has the responsibility and ability to address the bugs?

> Ideally, someone other than the spec Editor(s) should create the test cases
> (e.g. to avoid the "fox guarding the chicken coup").

The way I've been adding material to the new DOM Range spec is to
write simple tests to figure out how browsers work, then write a spec,
then write more tests based on the spec, then revise the spec if the
tests find it to be wrong.  I've found this essential when
reverse-engineering already-implemented features, because if I don't
write comprehensive tests then my spec will be wrong.  At the end of
the process, I naturally have lots and lots of tests, and it wouldn't
make sense to throw them out.

I do agree that all else being equal, it is better if the tests are
not written by the editor, to help catch spec vagueness where the
editor knows what they mean but no one else does.  But it might not
always work out that way.

Received on Sunday, 3 April 2011 22:32:46 UTC