RE: New tests submission: media element


> -----Original Message-----
> From: Simon Pieters [mailto:simonp@opera.com]
> Sent: Wednesday, September 12, 2012 4:20 PM
> To: public-html-testsuite@w3.org; Santos, Thiago; Zhang, Zhiqiang
> Cc: Carr, Wayne; Yu, Ling L
> Subject: Re: New tests submission: media element
> 
> On Wed, 12 Sep 2012 10:00:03 +0200, Zhang, Zhiqiang
> <zhiqiang.zhang@intel.com> wrote:
> 
> >> I don't have time to do a proper review right now, but I noticed that
> >> the
> >> general pattern of having the bulk of the testsuite in a single js file
> >> and then calling one function in the test itself, like:
> >>
> >>      2.19 +    <script type="text/javascript">
> >>      2.20 +        controls_attribute_type();
> >>      2.21 +    </script>
> >>
> >> ...makes it hard for the person reading the test to follow what it's
> >> doing. Generally it is recommended to put it inline in the test itself.
> >
> > Simon, thank you for your feedback.
> >
> > While inline JS code in each test file is good for reading the test,
> > putting common function in a single JS file and calling it in the test
> > is also a good practice, as
> > http://w3c-test.org/html/tests/approved/common/media.js

> >
> > As to these tests, I'll update them as your recommendation.
> 
> OK, thanks.
> 
> >> It seems the tests have more focus on describing what each test does and
> >> having boilerplate and links and everything than to make sure all
> >> aspects
> >> of a feature are covered (like error handling, edge cases, testing how
> >> multiple features interact). Bugs are more commonly found in the latter
> >> category, so tests that have that kind of coverage is more useful for
> >> the
> >> purpose of highlighting implementation bugs than tests that only check
> >> that shallow usage works.
> >
> > Currently, we are focusing on spec compliance tests, for features and
> > statements that are explicitly specified. Next, we'll be on multiple
> > features interaction, even features across specs.
> 
> Excellent!
> 
> > For the tests for error handling that the spec doesn't specify,
> 
> Error handling is specified.
> 
> > or for the specific implementation by different browsers/platforms,
> 
> What do you have in mind here?

Tests are based on the spec, not on the implementation :)

> 
> > I don't think they are suited to this test suite.
> 
> --
> Simon Pieters
> Opera Software

Received on Wednesday, 12 September 2012 08:29:31 UTC