Re: Fwd: Test Changes Proposal

On Thu, 16 Feb 2017 01:29:22 +0100, Benjamin Schaaf <>  

> I'm not really sure how we can tackle the user-agent defined margins
> for rendering. Perhaps an easy way to change it automatically
> throughout the tests would help?
> How do other tests handle this?

Yeah, we could have a single JS file that sets expected horizontal and  
vertical margins, where each vendor can put their own value, and that is  
used in the reference files to create the correct expected rendering...

We have some similar things, like /resources/testharnessreport.js which is  
empty but each vendor can put stuff there for integration of  
testharness.js with some other system.

> As for categorization, I just tested the help link you suggested and
> it doesn't seem to influence test outcome. So producing a report would
> involve parsing all the tests themselves (only a bit more work). But a
> categories.json may still be useful for more high-level reports, eg:
> "This browser passes 97% of the parsing tests for cues, but 0% of the
> parsing tests for regions"

OK. Yeah, maybe it doesn't affect anything in the test runner. Such links  
are used in CSS tests though for e.g. annotation the relevant  
specification with test results. Since WebVTT is generated with bikeshed  
(same as CSS specs), I think it should be possible for WebVTT as well. But  
it doesn't rule out having a JSON file as well. :-)

Simon Pieters
Opera Software

Received on Thursday, 16 February 2017 12:01:26 UTC