Re: Towards a better testsuite: Metadata

On 03/24/2016 01:00 PM, Geoffrey Sneddon wrote:
>
> I went through all of the all of the metadata in
> <https://lists.w3.org/Archives/Public/public-css-testsuite/2015Oct/0004.html>;
> there's rough agreement on removing much of what we currently have.
> That said, I think it's worthwhile to reiterate that requiring *any*
> metadata causes friction. Tests written by browser vendors are rarely
> a file or two which is quick to add metadata too. I know in general
> people seem interested in using the same infrastructure to run both
> web-platform-tests and csswg-test, which essentially requires the
> metadata required to run the tests be identical across the two.
>
> ...
>
> To outline what I'd like to see happen:
>
> - Get rid of the build system, replacing many of it's old errors with
> a lint tool that tests for them.
> - Policy changes to get rid of all metadata in the common case.
> - Change the commit policy. (Require review first, require no new lint errors.)

As I wrote in
   https://www.w3.org/mid/5702DA58.1020703@inkedblade.net
I agree with Florian's comments about metadata, and disagree
strongly with simply doing away with all of it. But I think
we can simplify things down to:

<!DOCTYPE html>
<title>Explain *exactly* what situation you're testing
and what it's supposed to do</title>
<link rel="help" href="spec">
<link rel="match" href="reference">

plus optional <meta name=flags> if needed for that test.
(I agree with trimming down the flags as well; many are
not really necessary at this point.)

This is short enough and straightforward enough that I think
anyone can learn to write a new test off the top of their
head after having written two or three. We can also have a
command line tool that generates the template, which is
something Mozilla already has built into its mach system for
writing WPT.


Wrt filenames, I would like us to recommend
   a) indexes be zero-filled to three places so tests sort correctly
   b) hyphens over underscore
but don't think we need to actually require anything here,
other than that it be descriptive of its content and not
obnoxiously long.

~fantasai

Received on Friday, 8 April 2016 17:01:01 UTC