W3C home > Mailing lists > Public > public-test-infra@w3.org > January to March 2017

Re: Test Changes Proposal

From: Benjamin Schaaf <ben.schaaf@gmail.com>
Date: Wed, 15 Feb 2017 10:33:14 +1100
Message-ID: <CAJ+kdVGJ0trmRztQm+vD_RVOhne_kf0hDajW2Ejg3m0yjdXNvw@mail.gmail.com>
To: Philip J├Ągenstedt <foolip@google.com>
Cc: public-test-infra@w3.org, Simon Pieters <simonp@opera.com>
I would change it to a format that includes both a vtt file and some
js out of which I can generate a test that adds the vtt file as a
track to a video element and passed the track and video to the js (and
run it).
That means I can write arbitrary assertions alongside the vtt file.

On Tue, Feb 14, 2017 at 11:22 PM, Philip J├Ągenstedt <foolip@google.com> wrote:
> Can you elaborate on how you'd change the parser tests? That seems like the
> only significant change. CC Simon in case he already replied about this
> before this was forwarded to the list.
>
>
> On Tue, Feb 14, 2017 at 2:43 PM Benjamin Schaaf <ben.schaaf@gmail.com>
> wrote:
>>
>> ---------- Forwarded message ----------
>> From: Benjamin Schaaf <ben.schaaf@gmail.com>
>> Date: Tue, Feb 14, 2017 at 12:48 PM
>> Subject: Test Changes Proposal
>> To: David Singer <singer@apple.com>, Simon Pieters <simonp@opera.com>,
>> Silvia Pfeiffer <silviapfeiffer1@gmail.com>
>>
>>
>> Hello,
>>
>> I've put together a proposal for a way to reorganise webvtt's
>> web-platform-tests to make writing and reading tests easier as well as
>> add categorization.
>> I'd like to get input on this before I start working on it.
>>
>> Thanks,
>> Benjamin Schaaf
>>
>>
>> # WebVTT web-platform-tests changes proposal
>>
>> A lot of the way tests are currently set up is limited in some manner, eg.
>> `webvtt-file-parsing tests` can only check direct attributes of webvtt
>> cues and
>> the number of cues generated. Instead of building on top of the current
>> tests,
>> I'd like to rewrite the way the current tests are run so we can more
>> easily
>> write tests.
>>
>> For `webvtt-file-parsing` I'd like to generate tests from a template that
>> includes a wevtt file and some js assertions given the video object.
>>
>> For `webvtt-cue-text-parsing-rules` I'd like to change the format to be
>> more
>> easily readable and clean up buildtests.py (or rewrite).
>>
>> I think the api tests are generally fine as they are (in terms of the way
>> tests
>> are run).
>>
>> I haven't dived deep enough into rendering tests to come up with anything
>> concrete, but there should be a nice way to combine a webvtt file, the
>> test file
>> and a ref file into a more easily writeable/readable single-file format
>> that we
>> can generate the rest from.
>>
>> ## Directory Structure
>>
>> I'd also like to clean up the directory structure. I'm not sure if any
>> deeper
>> directories are needed
>>
>> webvtt/
>>   api/
>>     VTTCue/
>>     VTTRegion/
>>   parsing/
>>     file-parsing/
>>     cue-text-parsing/
>>   rendering/
>>     TBD
>>
>> ## Categorization
>>
>> I propose we categorize tests by keeping track of the category of a test
>> separately (eg. in a `categories.json` file). We can then use the json
>> output of
>> the test runner to parse which categories/parts of the spec are well
>> supported.
>>
>> The tests for `2dcontext` do something similar.
>>
>
Received on Tuesday, 14 February 2017 23:33:48 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 17:34:13 UTC