Re: Test Changes Proposal

Perhaps a PR converting one test would make it easier to judge what this
would amount to. I guess that a problem with the current tests is that the
assertions are completely separate from the VTT file itself, but one
benefit is that the same VTT file can be shared among multiple tests.

On Wed, Feb 15, 2017 at 6:33 AM Benjamin Schaaf <ben.schaaf@gmail.com>
wrote:

> I would change it to a format that includes both a vtt file and some
> js out of which I can generate a test that adds the vtt file as a
> track to a video element and passed the track and video to the js (and
> run it).
> That means I can write arbitrary assertions alongside the vtt file.
>
> On Tue, Feb 14, 2017 at 11:22 PM, Philip Jägenstedt <foolip@google.com>
> wrote:
> > Can you elaborate on how you'd change the parser tests? That seems like
> the
> > only significant change. CC Simon in case he already replied about this
> > before this was forwarded to the list.
> >
> >
> > On Tue, Feb 14, 2017 at 2:43 PM Benjamin Schaaf <ben.schaaf@gmail.com>
> > wrote:
> >>
> >> ---------- Forwarded message ----------
> >> From: Benjamin Schaaf <ben.schaaf@gmail.com>
> >> Date: Tue, Feb 14, 2017 at 12:48 PM
> >> Subject: Test Changes Proposal
> >> To: David Singer <singer@apple.com>, Simon Pieters <simonp@opera.com>,
> >> Silvia Pfeiffer <silviapfeiffer1@gmail.com>
> >>
> >>
> >> Hello,
> >>
> >> I've put together a proposal for a way to reorganise webvtt's
> >> web-platform-tests to make writing and reading tests easier as well as
> >> add categorization.
> >> I'd like to get input on this before I start working on it.
> >>
> >> Thanks,
> >> Benjamin Schaaf
> >>
> >>
> >> # WebVTT web-platform-tests changes proposal
> >>
> >> A lot of the way tests are currently set up is limited in some manner,
> eg.
> >> `webvtt-file-parsing tests` can only check direct attributes of webvtt
> >> cues and
> >> the number of cues generated. Instead of building on top of the current
> >> tests,
> >> I'd like to rewrite the way the current tests are run so we can more
> >> easily
> >> write tests.
> >>
> >> For `webvtt-file-parsing` I'd like to generate tests from a template
> that
> >> includes a wevtt file and some js assertions given the video object.
> >>
> >> For `webvtt-cue-text-parsing-rules` I'd like to change the format to be
> >> more
> >> easily readable and clean up buildtests.py (or rewrite).
> >>
> >> I think the api tests are generally fine as they are (in terms of the
> way
> >> tests
> >> are run).
> >>
> >> I haven't dived deep enough into rendering tests to come up with
> anything
> >> concrete, but there should be a nice way to combine a webvtt file, the
> >> test file
> >> and a ref file into a more easily writeable/readable single-file format
> >> that we
> >> can generate the rest from.
> >>
> >> ## Directory Structure
> >>
> >> I'd also like to clean up the directory structure. I'm not sure if any
> >> deeper
> >> directories are needed
> >>
> >> webvtt/
> >>   api/
> >>     VTTCue/
> >>     VTTRegion/
> >>   parsing/
> >>     file-parsing/
> >>     cue-text-parsing/
> >>   rendering/
> >>     TBD
> >>
> >> ## Categorization
> >>
> >> I propose we categorize tests by keeping track of the category of a test
> >> separately (eg. in a `categories.json` file). We can then use the json
> >> output of
> >> the test runner to parse which categories/parts of the spec are well
> >> supported.
> >>
> >> The tests for `2dcontext` do something similar.
> >>
> >
>