Re: Fwd: Test Changes Proposal

Yea, I agree, having both (help and a JSON file) would be good. The
help links are good documentation and can help analyzing test coverage
and the JSON file can be used for classifying tests.

Maybe instead of just a js file for setting expected margins we can
have a JSON file that matches user agent strings to margins. This
would make it easy to run the tests on travis (since it runs the same
code for ff and chrome afaik).

On Thu, Feb 16, 2017 at 11:00 PM, Simon Pieters <simonp@opera.com> wrote:
> On Thu, 16 Feb 2017 01:29:22 +0100, Benjamin Schaaf <ben.schaaf@gmail.com>
> wrote:
>
>> I'm not really sure how we can tackle the user-agent defined margins
>> for rendering. Perhaps an easy way to change it automatically
>> throughout the tests would help?
>> How do other tests handle this?
>
>
> Yeah, we could have a single JS file that sets expected horizontal and
> vertical margins, where each vendor can put their own value, and that is
> used in the reference files to create the correct expected rendering...
>
> We have some similar things, like /resources/testharnessreport.js which is
> empty but each vendor can put stuff there for integration of testharness.js
> with some other system.
>
>> As for categorization, I just tested the help link you suggested and
>> it doesn't seem to influence test outcome. So producing a report would
>> involve parsing all the tests themselves (only a bit more work). But a
>> categories.json may still be useful for more high-level reports, eg:
>> "This browser passes 97% of the parsing tests for cues, but 0% of the
>> parsing tests for regions"
>
>
> OK. Yeah, maybe it doesn't affect anything in the test runner. Such links
> are used in CSS tests though for e.g. annotation the relevant specification
> with test results. Since WebVTT is generated with bikeshed (same as CSS
> specs), I think it should be possible for WebVTT as well. But it doesn't
> rule out having a JSON file as well. :-)
>
> http://web-platform-tests.org/writing-tests/css-metadata.html
>
>
> --
> Simon Pieters
> Opera Software

Received on Friday, 17 February 2017 02:22:08 UTC