Re: Calling All Tests

>
>
> However, I was more concerned about the reverse.  We have the WPT repo, so
> how do I get that into Chrome's testing infrastructure without modifying
> any test in the WPT repo? Without this, blink won't find any utility in the
> repo and will probably just continue to use it's existing test suite.
> Sadly.
>

I think that's an internal problem for browser vendors -- but assuming
there is some suitably easy way to invoke a WPT suite and extract a result
from it, then it shouldnt be hard.

If for some reason the WPT cannot be run that way, then I will be ready to
argue that we should use some framework that is easy for vendors to include
in their standard CI testing.



> I assume that despite the failures you can still call yourself an
> implementation of WebAudio that purports to conform.
>

I guess that's how the world works, unfortunately. But if there is an
agreed suite, it will at least get harder, and app developers are more
likely to call out implementations on these points: they can run the tests
too.


> My assumption here was that, say, FF found a bug in their implementation
> and want a test for that and pushes that to the repo. Which then causes
> Chrome to fail that test (for whatever reason).  We wouldn't want to block
> FF from having a regression test, but we wouldn't want to be blocked from
> updating because the test will cause failures in our test bots.
>

Perhaps we need a way to gate tests in the suite so that you are able to
arbitrarily exclude ones that are not applicable for some reason (or
arbitrarily include the ones you want).

I won't have time to do this myself in the near future, but I would like to
> pull a few tests over from the WPT repo and run them to see how they would
> work in Chrome.   Perhaps everything will work just fine.
>

Thanks -- It'd be great to understand this landscape better.

...Joe