RE: New tests submission: media element


> -----Original Message-----
> From: Simon Pieters [mailto:simonp@opera.com]
> Sent: Tuesday, September 11, 2012 8:16 PM
> To: public-html-testsuite@w3.org; Santos, Thiago
> Cc: Carr, Wayne; Zhang, Zhiqiang
> Subject: Re: New tests submission: media element
> 
> On Tue, 11 Sep 2012 12:47:59 +0200, Thiago Marcos P. Santos
> <thiago.santos@intel.com> wrote:
> 
> > Hello,
> >
> > I would like to submit the following tests:
> > http://dvcs.w3.org/hg/html/rev/dcbfd1474a19

> >
> > We are verifying the conformance with the following specs:
> > http://dev.w3.org/html5/spec/single-page.html#dom-media-controls

> > http://dev.w3.org/html5/spec/single-page.html#dom-media-

> defaultmuted
> > http://dev.w3.org/html5/spec/single-page.html#dom-media-loop

> > http://dev.w3.org/html5/spec/single-page.html#dom-media-muted

> > http://dev.w3.org/html5/spec/single-page.html#dom-media-volume

> > http://dev.w3.org/html5/spec/single-page.html#dom-dim-height

> > http://dev.w3.org/html5/spec/single-page.html#dom-dim-width

> >
> > Review is appreciated and Zhiqiang Zhang will address any feedback.
> 
> I don't have time to do a proper review right now, but I noticed that the
> general pattern of having the bulk of the testsuite in a single js file
> and then calling one function in the test itself, like:
> 
>      2.19 +    <script type="text/javascript">
>      2.20 +        controls_attribute_type();
>      2.21 +    </script>
> 
> ...makes it hard for the person reading the test to follow what it's
> doing. Generally it is recommended to put it inline in the test itself.

Simon, thank you for your feedback.

While inline JS code in each test file is good for reading the test, putting common function in a single JS file and calling it in the test is also a good practice, as http://w3c-test.org/html/tests/approved/common/media.js


As to these tests, I'll update them as your recommendation.

> 
> It seems the tests have more focus on describing what each test does and
> having boilerplate and links and everything than to make sure all aspects
> of a feature are covered (like error handling, edge cases, testing how
> multiple features interact). Bugs are more commonly found in the latter
> category, so tests that have that kind of coverage is more useful for the
> purpose of highlighting implementation bugs than tests that only check
> that shallow usage works.

Currently, we are focusing on spec compliance tests, for features and statements that are explicitly specified. Next, we'll be on multiple features interaction, even features across specs. For the tests for error handling that the spec doesn't specify, or for the specific implementation by different browsers/platforms, I don't think they are suited to this test suite.

> 
> cheers
> --
> Simon Pieters
> Opera Software

Received on Wednesday, 12 September 2012 08:00:48 UTC