Re: Test Suite Language

I think that good tests have a transparent relationship between the documented intent of the test to what the test is doing.  That suggests minimizing the number of translation steps, but it’s not a showstopper if there are translation steps.

If the tests start out as C++, why not compile to WebAssembly?  Is compiling to JavaScript better in some way?

-Filip


> On Sep 28, 2018, at 2:21 PM, Myles C. Maxfield <mmaxfield@apple.com> wrote:
> 
> Hello, all
> 
> Yesterday, we spent a while discussing which language the WebGPU test suite should be written in. I thought I’d clarify some of our thoughts, as I’m not sure they came through clearly in the discussion.
> 
> Having a compilation step for our test suite is completely acceptable. Nowadays, even documentation has compile steps, so it seems completely reasonable for us to have one here as well. A compile step that compiles a C++ program and runs it to produce Javascript code, is still just a compile step. This is fine even if the structure of the produced Javascript isn’t perfectly faithful to the original source code (aka a “trace”).
> 
> We do have some desires for this test suite, though. First, we hope that the test suite can be hosted on a web server, so users can run it with a single tap/click (we feel this way about the WebGL test suite too). Secondly, if there is a compilation step, it shouldn’t require an incredibly complicated machine setup in order to build. Lastly, and perhaps most importantly, the debugging experience for the Javascript tests shouldn’t be too arduous. At the very least, we’d like to step-over individual calls using Web Inspector, rather than having the test represented in a JSON structure, for example. If we’re going to pursue the compilation-that-produces-javascript strategy, we’d like to take a look at a few examples of the produced Javascript before ramping up test production, to see how difficult it would be to debug. There were also some concerns yesterday that a trace may not be able to accurately capture the intent of most tests, so that claim either needs to be demonstrated as false, or some other Javascript structure should be produced.
> 
> Given these thoughts, I hope we can reach a compromise.
> 
> Thanks,
> Myles and Dean

Received on Friday, 28 September 2018 22:22:19 UTC