Re: Proposed policy change: reusability of tests by other browsers

CC +public-test-infra

On 10/10/2012 08:08 PM, Boris Zbarsky wrote:
> On 10/10/12 4:26 AM, james@hoppipolla.co.uk wrote:
>> Do you have a concrete suggestion for how to improve this, in a way
>> that works cross-browser?
>
> Well, the test harness could include an HTTP server.  The Mozilla
> harness happens to use one written in JS, but one could be done in
> python instead, say.

Hmm, so I'm not quite sure how that would work. Is the system that you 
are imagining one where the tests (and the harness) are all installed 
locally on the device being tested, or, at least, that there is a 1:1 
ratio between server instances and test machines? That isn't the setup 
that we have traditionally used (we have had one ordinary apache server 
serving tests, which — unfortunately — are not versioned with the code). 
We could probably adapt to it somehow, although great ways to set it up 
are eluding me at the moment.

I also don't see how such a system would meet the needs of people that 
want to run tests from w3c-test.org in a simple web-based clients 
without having substantial local setup (e.g. the Core Mobile people).

However, I am not trying to be dismissive of this proposal; if there is 
something that we can make work here that addresses all the uses cases, 
or at least more use cases than are addressed today, we should do it for 
sure.

What exactly are the requirements that you have? Presumably it should be 
possible to invoke custom server behaviour for some tests? How should 
this work? How do you deal with things like https?

Received on Thursday, 11 October 2012 09:21:06 UTC