Re: Towards a better testsuite

On 28/03/16 04:47, Florian Rivoal wrote:
> As far as the path of the file carying information about the spec it is
> testing, that's sort of true, but it doesn't indicate which part of a
> spec, which is useful, and it doesn't help when testing multiple specs
> at a time.

In wpt the path does indicate what section of the spec it is testing:
there's far more levels of directories there than there are in
csswg-test, and I think that's a good thing, as it makes it far easier
to find tests.

>> For wpt we have master on w3c-test.org <http://w3c-test.org/>, though
>> that doesn't quite suffice for running them all given some rely on
>> specific host names resolving. It's also not the easiest way to run
>> reftests, but I'm not sure what can be done to make that easier:
>> determining pixel by pixel equivalence by eye will always be hard (and
>> really, quickly changing between tabs/frames can easily end up with
>> things being incorrectly marked, it only really makes sense to do
>> programmatically).
>>
> Having it on the master branch mean you can't use it for reviews.

PRs from a whitelisted set of users get automatically mirrored to
http://w3c-test.org/submissions/ and PRs from other users can be added
to that by any of the whitelisted users (there's arbitrary code
execution going on server-side, so we can't just throw up any PR).

/Geoffrey

Received on Monday, 28 March 2016 20:27:06 UTC