Re: Calling All Tests

Hi Joe and all,

Sonoport was also keen on helping out with this effort. Unfortunately we
couldn't get a suitable intern this summer to work on this. However, I
would be interested in contributing to some extent.

I did spend a little time looking through the various test suites
(WebPlatform tests, Browser Vendor tests, and the web-audio-test-api
library). I had been planning for the following initial tasks.

- Clean up IDL based WebPlatform tests
- Add tests for all the other Nodes
- Try to integrate tests from Various Browser platforms
- Explore CI based automated testing

I propose this could be done collaboratively on Github, so individual
issues could be taken on by various people who're willing to contribute to
this.

-Chinmay


On Tue, Jun 9, 2015 at 6:12 PM, Ryoya KAWAI <ryoya.kawai@gmail.com> wrote:

> He, @mohayonao, is one of the Japanese developer who builds great Web
> Audio applications. He might in this list, but I can ask him to help.
>
> --
>  Ryoya KAWAI : ryoya.kawai@gmail.com
>
>
> On Tue, Jun 9, 2015 at 6:59 PM, Matthew Paradis
> <matthew.paradis@bbc.co.uk> wrote:
> > As mentioned by Paul at the F2F there is also a test suite in progress
> here
> >
> > https://github.com/mohayonao/web-audio-test-api/
> >
> > Matt
> >
> > --
> > Matthew Paradis
> > Senior Software Engineer (Audio),
> > BBC Research & Development
> > 030304 09889 | matthew.paradis@bbc.co.uk | www.bbc.co.uk/rd/sound
> >
> > From: Joe Berkovitz <joe@noteflight.com>
> > Date: Monday, 8 June 2015 22:06
> > To: Raymond Toy <rtoy@google.com>
> > Cc: Chris Lowis <chris.lowis@gmail.com>, Paul Adenot <
> padenot@mozilla.com>,
> > Audio Working Group <public-audio@w3.org>
> > Subject: Re: Calling All Tests
> > Resent-From: <public-audio@w3.org>
> > Resent-Date: Monday, 8 June 2015 22:06
> >
> >>
> >> However, I was more concerned about the reverse.  We have the WPT repo,
> so
> >> how do I get that into Chrome's testing infrastructure without
> modifying any
> >> test in the WPT repo? Without this, blink won't find any utility in the
> repo
> >> and will probably just continue to use it's existing test suite.  Sadly.
> >
> >
> > I think that's an internal problem for browser vendors -- but assuming
> there
> > is some suitably easy way to invoke a WPT suite and extract a result from
> > it, then it shouldnt be hard.
> >
> > If for some reason the WPT cannot be run that way, then I will be ready
> to
> > argue that we should use some framework that is easy for vendors to
> include
> > in their standard CI testing.
> >
> >
> >>
> >> I assume that despite the failures you can still call yourself an
> >> implementation of WebAudio that purports to conform.
> >
> >
> > I guess that's how the world works, unfortunately. But if there is an
> agreed
> > suite, it will at least get harder, and app developers are more likely to
> > call out implementations on these points: they can run the tests too.
> >
> >>
> >> My assumption here was that, say, FF found a bug in their implementation
> >> and want a test for that and pushes that to the repo. Which then causes
> >> Chrome to fail that test (for whatever reason).  We wouldn't want to
> block
> >> FF from having a regression test, but we wouldn't want to be blocked
> from
> >> updating because the test will cause failures in our test bots.
> >
> >
> > Perhaps we need a way to gate tests in the suite so that you are able to
> > arbitrarily exclude ones that are not applicable for some reason (or
> > arbitrarily include the ones you want).
> >
> >> I won't have time to do this myself in the near future, but I would like
> >> to pull a few tests over from the WPT repo and run them to see how they
> >> would work in Chrome.   Perhaps everything will work just fine.
> >
> >
> > Thanks -- It'd be great to understand this landscape better.
> >
> > ...Joe
> >
> >
> >
> > ----------------------------
> >
> > http://www.bbc.co.uk
> > This e-mail (and any attachments) is confidential and may contain
> personal
> > views which are not the views of the BBC unless specifically stated.
> > If you have received it in error, please delete it from your system.
> > Do not use, copy or disclose the information in any way nor act in
> reliance
> > on it and notify the sender immediately.
> > Please note that the BBC monitors e-mails sent or received.
> > Further communication will signify your consent to this.
> >
> > ---------------------
>
>