Re: UserAgent-specific files in Web Platform Tests

Thanks for sharing that, Attila, I had no idea about this project in Servo,
but it's great that the tests provided value.

It is actually not too surprising that the tests were copied out of the
Chromium tree, something happened with Service Worker tests although in
that case they were copied into web-platform-tests and currently there's
work ongoing to reconcile them. Over time, something similar might happen
with WebBluetooth if there's any friction in keeping the tests in sync.

James, I understand that you still have concerns with this approach, and
there are things that might go wrong. However, I would suggest that the
value of sharing tests is potentially high, and that having forked test
suites is no fun. web-platform-tests would be the main test suite for
WebBluetooth in Chrome, so just like for the spec itself, there should
always be someone who can answer questions on the Chromium side.

If we upstream these tests, I think the following would be reasonable:

   - A README that clearly documents the requirements for running the tests.
   - Intelligible failure messages when testing APIs aren't present.
   - A Chromium tracking bug
   <https://bugs.chromium.org/p/chromium/issues/detail?id=725105> to get
   the tests running on vanilla Chrome binaries and working on PR stability
   testing and the upcoming wpt dashboard.

Do you think that would suffice?

Thanks!

On Sat, May 20, 2017 at 12:09 AM Attila Dusnoki <dati91@gmail.com> wrote:

> Hi All,
>
> I'm part of a small team at the University of Szeged who implemented
> the WebBluetooth spec for Servo. We already use Chrome's Layout Tests
> for our testing, based on a previous Testing API proposal.
> Those wpt tests have been already proved their value catching many
> bugs during the implementation, and the newly proposed Testing API
> looks even more promising.
> In our case the testing api is behind a build flag and the
> WebBluetooth API is behind a runtime flag.
> Since the Servo is an experimental browser, it is much more easier to
> introduce and implement new features compared to any other well-known
> browser. So, we do not have any strong opinion on any side about the
> previously discussed topics. The only thing we would like to mention
> that the current wpt tests are valuable for us.
>
> So, we are looking forward to see the newly proposed WebBluetooth
> Testing API getting to a stable version.
>
> Regards,
> Attila
>