Re: Review of tests upstreamed by implementors

On Wed, Mar 20, 2013 at 2:45 PM, Tobie Langel <tobie@w3.org> wrote:

> On Wednesday, March 20, 2013 at 10:29 PM, Rebecca Hauck wrote:
> > And you're probably right about
> > implementors not running all or many of the W3C tests. The quandary is
> > that implementors seem to want to only take tests (not authored by them)
> > that are blessed by the W3C and the W3C wants implementors to bless the
> > tests by running them.
>
>  One of my goals is to make sure that we automatically run the tests on a
> number of different user agents both on the master branch and on incoming
> pull requests.
>
>
While I do not doubt that this would be useful, I'm not sure it will be as
useful as you might like, nor as easy. Perhaps I'm not accurately picturing
what you have in mind: I'm taking this to mean you hope to be running
patches on your own infrastructure? Maintaining such an infrastructure is
likely a lot of work (even keeping Chromium and WebKit's internal
infrastructures for building and testing functional is a full-time job;
multiple user agents is probably more than linearly worse).

I think I can speak fairly confidently that, at least from Chromium's point
of view, until you/we actually get the tests integrated into our existing
continuous build systems, you're/we're always going to be fighting for
attention. I am at least hopeful that we can avoid this fight. I suspect
this is probably true for most of the other WebKit ports and Mozilla,
although of course I can't speak for them.

-- Dirk