Fwd: Test Changes Proposal

---------- Forwarded message ----------
From: Benjamin Schaaf <ben.schaaf@gmail.com>
Date: Tue, Feb 14, 2017 at 12:48 PM
Subject: Test Changes Proposal
To: David Singer <singer@apple.com>, Simon Pieters <simonp@opera.com>,
Silvia Pfeiffer <silviapfeiffer1@gmail.com>


Hello,

I've put together a proposal for a way to reorganise webvtt's
web-platform-tests to make writing and reading tests easier as well as
add categorization.
I'd like to get input on this before I start working on it.

Thanks,
Benjamin Schaaf


# WebVTT web-platform-tests changes proposal

A lot of the way tests are currently set up is limited in some manner, eg.
`webvtt-file-parsing tests` can only check direct attributes of webvtt cues and
the number of cues generated. Instead of building on top of the current tests,
I'd like to rewrite the way the current tests are run so we can more easily
write tests.

For `webvtt-file-parsing` I'd like to generate tests from a template that
includes a wevtt file and some js assertions given the video object.

For `webvtt-cue-text-parsing-rules` I'd like to change the format to be more
easily readable and clean up buildtests.py (or rewrite).

I think the api tests are generally fine as they are (in terms of the way tests
are run).

I haven't dived deep enough into rendering tests to come up with anything
concrete, but there should be a nice way to combine a webvtt file, the test file
and a ref file into a more easily writeable/readable single-file format that we
can generate the rest from.

## Directory Structure

I'd also like to clean up the directory structure. I'm not sure if any deeper
directories are needed

webvtt/
  api/
    VTTCue/
    VTTRegion/
  parsing/
    file-parsing/
    cue-text-parsing/
  rendering/
    TBD

## Categorization

I propose we categorize tests by keeping track of the category of a test
separately (eg. in a `categories.json` file). We can then use the json output of
the test runner to parse which categories/parts of the spec are well supported.

The tests for `2dcontext` do something similar.

Received on Tuesday, 14 February 2017 07:42:41 UTC