Re: Getting vendors using W3C tests

On Tue, May 31, 2011 at 6:52 AM, James Graham <jgraham@opera.com> wrote:
> * Getting the results out of the test into our systems. In theory this is
> solved for testharness.js tests by including a second <script>;
> testharnessreport.js that is blank on the W3C site and which vendors can use
> to convert the testharness.js results to whatever is needed
> result-collection system they use. However not all tests have consistently
> included this extra script.

I never got the point of that.  Why don't reusers just hack
testharness.js itself?  Or if we want it to be a separate file for
convenience, why doesn't testharness.js add the <script> to include
that file automatically?

> The second problem needs a bit of thought. Possibly the testsuite
> could have a build step and servers could be variables in the original
> files. That has some disadvantages though since there is non-negligible
> overhead to making everyone build the testsuite before they can run it.

If we make sure that the set of hosts is fixed and their roles are
well-documented, would it be unreasonable to ask implementers to just
edit the host file on their test machine to point the domain to
someplace they control?  That seems like it would be a lot simpler
than any of the alternatives I can think of.

Received on Tuesday, 31 May 2011 22:27:22 UTC