Re: Getting browser vendors running and submitting tests

On 10/28/2015 05:21 PM, Geoffrey Sneddon wrote:
> [CC'd to public-test-infra; this is really mostly about CSS, can we avoid cross-posting replies and keeping them on
> public-css-testsuite?]
>
> Myself and fantasai chaired a session at TPAC earlier today on this subject. The minutes are at
> <http://www.w3.org/2015/10/28-testing-minutes.html> (thanks fantasai!).
>
> Notably:
>
> * The build system makes it difficult for vendors to upstream their fixes and their own tests (as vendors can't just fix the
> files they run; they have to find some other file, fix that, and then work out how to build the test suite). As a result, they
> just fix the built files and don't upstream the fixes and often avoid updating the copy of the test suite because it would
> trample their fixes. We should get rid of the build system (at least insofar as it modifies/moves files), as I don't think
> anyone likely to run the tests doesn't support HTML at this point (yay the HTML5 parser!).
>
> * We should get rid of metadata, at least in the common case, required to submit tests to the the test suite, as this is an
> impediment to upstreaming tests. (See also my recent thread on public-css-testsuite, "Simplifying metadata".)

I disagree with this. I think we should reduce the metadata,
but there are some things (e.g. spec section associations)
that we need to keep.

~fantasai