Re: Specifications with matching test APIs

Thanks Reilly, I support continuing to move in this direction!

Are you planning in shipping this API in release Chrome builds behind a
flag?  I think it would be reasonable to modify the WPT infrastructure
(stability_checker, dashboard) to pass a --enable-testing-apis flag
(although we might need to consider the security implications of that for
the WPT infrastructure running potentially untrusted test patches).  But I
don't think we'd want to use content_shell (or even Chromium builds) in
that infrastructure - at least not in place of Chrome builds.

What's your thoughts on writing tests which could be either manual tests
("attach a device with the following characteristics, etc..") or automated
tests?

We could probably bikeshed a bit on the spec terminology here. "At that
point the behavior of usb is defined by this specification" seems a little
wrong in principle to me.  We want web-platform-tests to be testing the Web
USB spec itself - so test cases should be written against the algorithms
defined there (not in a spec for a feature that technically doesn't ship to
users).  But since the testing spec then refers back to running algorithms
in the Web USB spec maybe that's OK?


On Sat, Apr 22, 2017 at 9:56 AM, Reilly Grant <reillyg@chromium.org> wrote:

> On Fri, Apr 21, 2017 at 11:00 AM Reilly Grant <reillyg@chromium.org>
> wrote:
>
>> As a followup to this thread
>> <https://lists.w3.org/Archives/Public/public-test-infra/2017AprJun/0013.html> on
>> UserAgent-specific files in Web Platform Tests I would like to draw this
>> group's attention to the WebUSB Testing API
>> <https://wicg.github.io/webusb/test/>. This specification formally
>> defines an additional API that can be implemented by a UA to permit
>> automated testing of hard-to-test APIs such as WebUSB and Web Bluetooth.
>> Some example tests are included.
>>
>
> It was pointed out to me that my mention of Web Bluetooth here may be
> confusing. The WebUSB Testing API is an *example* of the kind of API that
> could be used for hard-to-test APIs like Web Bluetooth. This particular API
> is only useful for testing WebUSB.
>