Re: Test Case Template/Meta Data

On Thu, Nov 19, 2009 at 2:50 AM, Kris Krueger <krisk@microsoft.com> wrote:
> I'd like to get agreement on a test case template for the HTML testing task force.
> To start with I'd like to propose that the template allow for including other key information about the test using a meta tag.
>
> For example a test case that depends upon the ahem font would contain.
>
>    <meta name="flags" content="ahem" />
>
> Other key information that may been necessary
>
>    Author       -> Who created the test
>    Status/Phase -> Approved/Draft/Submitted
>    Reviewer     -> Who reviewed the test case
>    Help         -> URI back to the specification
>    Assert       -> Describes what specifically the test case tests
>
> Thoughts?  Other additional information that we would like to have?

I wrote a document some times ago about a possible cross browser test
format: http://omocha.w3.org/wiki/newformat
It could serve as a starting point for a test case template for the testsuite.

I agree that such a format is not meant to be the format that fits
everyone, and that some other existing ad hoc formats are more
appropriate for some kind of tests.

For instance, I see a common format useful for:
 - converting existing non automated tests. That's the tests that
print a PASS/FAIL message for instance.
 - converting existing tests from browser vendors. WebKit and Mozilla
have already lots of test, but their format sometimes relies on
browser specific features or environment.
 - or simply new tests if they don't have needs requiring some more
custom formats.

Regarding metadata, I simply reused elements from the CSS Test suite
Test format (http://wiki.csswg.org/test/css2.1/format#template-details),
which seem to match the information you propose here.

Reading the thread here, I see that some information could be dropped
(author, assert for instance). Or it could be optional and stored
externally in a DB.

Other than metadata, I think the other important elements of such a
format are the JavaScript API for assertions and specifying the
testing environment (for things such as sending custom headers,
dynamic content and so forth).


Sylvain

Received on Friday, 20 November 2009 21:39:45 UTC