Re: Test Changes Proposal

That is true, neither of those have payloads where the expectations can be
put, although for the existing parser tests I assume that only really works
for the key:value things trailing the timing, and not parsing of the cue
text itself.

Maybe a new test for stylesheet parsing using your suggested approach?

On Wed, Feb 15, 2017 at 12:41 PM Benjamin Schaaf <ben.schaaf@gmail.com>
wrote:

> For the file-parser tests its currently one test per vtt file, and any
> assertions are either in the vtt file (as attributes for a `VTTCue`
> object, eg: `{"line":0}`) or in cue-counts.json (which just assets
> that a specific number of cues were parsed).
> That doesn't easily expand into tests for stylesheet or VTTRegion parsing.
>
> On Wed, Feb 15, 2017 at 4:37 PM, Philip Jägenstedt <foolip@google.com>
> wrote:
> > Perhaps a PR converting one test would make it easier to judge what this
> > would amount to. I guess that a problem with the current tests is that
> the
> > assertions are completely separate from the VTT file itself, but one
> benefit
> > is that the same VTT file can be shared among multiple tests.
> >
> > On Wed, Feb 15, 2017 at 6:33 AM Benjamin Schaaf <ben.schaaf@gmail.com>
> > wrote:
> >>
> >> I would change it to a format that includes both a vtt file and some
> >> js out of which I can generate a test that adds the vtt file as a
> >> track to a video element and passed the track and video to the js (and
> >> run it).
> >> That means I can write arbitrary assertions alongside the vtt file.
> >>
> >> On Tue, Feb 14, 2017 at 11:22 PM, Philip Jägenstedt <foolip@google.com>
> >> wrote:
> >> > Can you elaborate on how you'd change the parser tests? That seems
> like
> >> > the
> >> > only significant change. CC Simon in case he already replied about
> this
> >> > before this was forwarded to the list.
> >> >
> >> >
> >> > On Tue, Feb 14, 2017 at 2:43 PM Benjamin Schaaf <ben.schaaf@gmail.com
> >
> >> > wrote:
> >> >>
> >> >> ---------- Forwarded message ----------
> >> >> From: Benjamin Schaaf <ben.schaaf@gmail.com>
> >> >> Date: Tue, Feb 14, 2017 at 12:48 PM
> >> >> Subject: Test Changes Proposal
> >> >> To: David Singer <singer@apple.com>, Simon Pieters <simonp@opera.com
> >,
> >> >> Silvia Pfeiffer <silviapfeiffer1@gmail.com>
> >> >>
> >> >>
> >> >> Hello,
> >> >>
> >> >> I've put together a proposal for a way to reorganise webvtt's
> >> >> web-platform-tests to make writing and reading tests easier as well
> as
> >> >> add categorization.
> >> >> I'd like to get input on this before I start working on it.
> >> >>
> >> >> Thanks,
> >> >> Benjamin Schaaf
> >> >>
> >> >>
> >> >> # WebVTT web-platform-tests changes proposal
> >> >>
> >> >> A lot of the way tests are currently set up is limited in some
> manner,
> >> >> eg.
> >> >> `webvtt-file-parsing tests` can only check direct attributes of
> webvtt
> >> >> cues and
> >> >> the number of cues generated. Instead of building on top of the
> current
> >> >> tests,
> >> >> I'd like to rewrite the way the current tests are run so we can more
> >> >> easily
> >> >> write tests.
> >> >>
> >> >> For `webvtt-file-parsing` I'd like to generate tests from a template
> >> >> that
> >> >> includes a wevtt file and some js assertions given the video object.
> >> >>
> >> >> For `webvtt-cue-text-parsing-rules` I'd like to change the format to
> be
> >> >> more
> >> >> easily readable and clean up buildtests.py (or rewrite).
> >> >>
> >> >> I think the api tests are generally fine as they are (in terms of the
> >> >> way
> >> >> tests
> >> >> are run).
> >> >>
> >> >> I haven't dived deep enough into rendering tests to come up with
> >> >> anything
> >> >> concrete, but there should be a nice way to combine a webvtt file,
> the
> >> >> test file
> >> >> and a ref file into a more easily writeable/readable single-file
> format
> >> >> that we
> >> >> can generate the rest from.
> >> >>
> >> >> ## Directory Structure
> >> >>
> >> >> I'd also like to clean up the directory structure. I'm not sure if
> any
> >> >> deeper
> >> >> directories are needed
> >> >>
> >> >> webvtt/
> >> >>   api/
> >> >>     VTTCue/
> >> >>     VTTRegion/
> >> >>   parsing/
> >> >>     file-parsing/
> >> >>     cue-text-parsing/
> >> >>   rendering/
> >> >>     TBD
> >> >>
> >> >> ## Categorization
> >> >>
> >> >> I propose we categorize tests by keeping track of the category of a
> >> >> test
> >> >> separately (eg. in a `categories.json` file). We can then use the
> json
> >> >> output of
> >> >> the test runner to parse which categories/parts of the spec are well
> >> >> supported.
> >> >>
> >> >> The tests for `2dcontext` do something similar.
> >> >>
> >> >
>