Re: Automating testcases

On 09/05/2010 09:29 AM, Geoffrey Sneddon wrote:
> Hey,
>
> There was a brief discussion at the F2F a couple of weeks back about
> automating tests within the CSS 2.1 testsuite, and although several
> vendors seemed to be interested in helping with such an effort, I
> remember some comment being made by one of the Microsoft reps that they
> didn't want to convert them to reftests (or at least didn't intend on
> spending any effort on doing so; I do remember this being a bit
> ambiguous at the time…).
>
> However, the first large set of tests that it makes sense to automate
> are tests submitted by Microsoft: there are 400–500 tests for which the
> reference is effectively just "<p>Test passes if there is no red visible
> on the page.</p>". This is the largest collection of tests in the
> testsuite with a single reference.
>
> If we (i.e., the browser vendors apart from MS represented at the
> meeting, Apple, Google, Mozilla, and Opera) wish to convert them to
> reftests, how should we go about this?

Make a file called ref-no-red.xht, and then draw up some reftest manifests
for each of Microsoft's directories matching up each test to that reference.

Add the line
   push @dirs, 'contributors/microsoft/submitted/Chapter*/reftest.list';
to tools/pub-css21.pl right before the existing line that adds the selftests.
(I think that should be sufficient to add to the build, though I'd have to
test it to be sure.)

Then send the patch as an attachment to Arron, CCing this list, and ask for
review. If he says it's okay to check in, then check it in. (If not, he should
cite reasons why it shouldn't be checked in and you should fix them then
resubmit for review.)

~fantasai

Received on Tuesday, 7 September 2010 19:04:10 UTC