Re: Tests for video, track and htmlmediaelement

On 05/23/2013 05:15 PM, Bob Lund wrote:
> Hi,
>
> We've got a small team that intends to generate and contribute tests for
> HTML5 video, track and media elements. I want to share our plan and I've
> got a few questions.
>
> Robin suggested that it would be most helpful to first sort through the
> test backlog[1][2] to see what applies to the video, track and media
> spec sections; we will do this. Is there a tool or method people are
> using to correlate tests with specific spec text so that we can track
> spec coverage?

At the highest level, the correspondence of tests to sections is tracked 
by the directory that the test is in in the repo. The directory heirachy 
corresponds to the ids of the headings in the spec. There is also the 
possibility to add metadata in the test files themselves; see [1], [2] 
for documentation.

I think Robin has some tools to visualise coverage based on this data.

Note that although "high coverage" is a good start, it is not a good 
end; lots of features interact with other features in slightly complex 
ways so merely having a test for some part of the spec doesn't mean that 
a useful fraction of the behaviour implied by that spec text is covered.

> BTW, I'm assuming that [3] is the proper spec version.
> Has anyone correlated the accepted tests for media[4], video[5] and
> track[6] to spec text? Are there any other test repositories we should
> be looking at? As we identify tests that apply, what's the procedure to
> get them reviewed?

Test review happens via pull requests. One of the most useful things you 
could do would be to provide review of existing tests, both for 
correctness and for conforming to the idioms of the testsuite (e.g. in 
the way that they use testharness.js). You can either do this through 
github or by using Critic (a code review tool developed for use at 
Opera); there is a critic instance at https://critic.hoppipolla.co.uk 
which should have one review per open pull request. Log in is via github 
using oAuth.

If you have any further questions please ask, either here or on IRC in 
#testing on irc.w3.org or #whatwg on freenode.

[1] http://wiki.csswg.org/test/format#specification-links
[2] http://wiki.csswg.org/test/scripttest#per-test-metadata

Received on Thursday, 23 May 2013 16:02:03 UTC