Per-test metadata support for script tests

Hello all,

I just landed some changes to testharness.js and testharnessreport.js to add support for per-test metadata.

You can now optionally pass spec links, assertion text, and author information in the test constructor by adding them to the properties parameter. Each value can be a string or an array of strings, i.e.:

test(function() { assert_true(true); }, 'test_name', 
                 { help: 'http://www.w3.org/TR/spec#section', 
                   assert: ['this tests something'., 'This also tests something else.'],
                   author: 'John Doe <john@doe.com>' });

These values would override any metadata set in the <head> of the test and are only needed when the individual test's metadata is different than what's in the <head>.

In addition, in order to allow the metadata to be read by non-browser processors (like Shepherd or for preparing manifest files for the test framework), I've added a convention of storing a cached version of the per-test metadata encoded in JSON in the <head>.

The default version of testharnessreport.js now contains code that reads your per-test metadata and compares it to the cached version. If the cache is not present, or is out of sync, it'll display a message to that effect and generate a link which, when clicked, will generate the appropriate source code for the cached metadata suitable for copy/paste into the test's <head>. The metadata cache need only be generated once just before submitting a test to suites that care about metadata, and regenerated only when the metadata changes. I tried to keep it unobtrusive and simple for the test authors, feedback to improve it is welcome.

Note that if there's only a single script test in a file, the metadata cache is not required (and in fact, should not be present) as all the appropriate metadata should be in the <head> to begin with.

This also sets the expectation that test names should be short, concise, unique and stable (i.e. they should not change as tests get added/removed). Longer human readable descriptions of the test should be stored in the 'assert' metadata. This will come into play more as I add support for multiple test script tests to the test framework.

I also added the per-test assertion text to the generated results table when present.

Peter

Received on Thursday, 14 June 2012 02:42:54 UTC