Re: UserAgent-specific files in Web Platform Tests

On 22/05/17 17:13, Philip Jägenstedt wrote:
> Thanks for sharing that, Attila, I had no idea about this project in 
> Servo, but it's great that the tests provided value.
> 
> It is actually not too surprising that the tests were copied out of the 
> Chromium tree, something happened with Service Worker tests although in 
> that case they were copied into web-platform-tests and currently there's 
> work ongoing to reconcile them. Over time, something similar might 
> happen with WebBluetooth if there's any friction in keeping the tests in 
> sync.
> 
> James, I understand that you still have concerns with this approach, and 
> there are things that might go wrong. However, I would suggest that the 
> value of sharing tests is potentially high, and that having forked test 
> suites is no fun. web-platform-tests would be the main test suite for 
> WebBluetooth in Chrome, so just like for the spec itself, there should 
> always be someone who can answer questions on the Chromium side.
> 
> If we upstream these tests, I think the following would be reasonable:
> 
>   * A README that clearly documents the requirements for running the tests.
>   * Intelligible failure messages when testing APIs aren't present.
>   * A Chromium tracking bug
>     <https://bugs.chromium.org/p/chromium/issues/detail?id=725105> to
>     get the tests running on vanilla Chrome binaries and working on PR
>     stability testing and the upcoming wpt dashboard.
> 
> Do you think that would suffice?

So, I still think this general approach is likely to be problematic in 
the end. But I don't object to upstreaming these tests if there are 
actually multiple implementations using them, subject to completion of 
the above action items, with the proviso that I won't be surprised if we 
revisit this problem in the future and end up deciding that a different 
tack is necessary. In particular I don't think we should use this as a 
precedent for future testsuites to follow; I don't think it's a "good 
enough" solution in general — and especially not in cases where there is 
only a Chrome implementation — but I do think that upstreaming them is 
more helpful than Servo pulling them out of the Blink repo.

Received on Tuesday, 23 May 2017 12:37:11 UTC