W3C home > Mailing lists > Public > public-test-infra@w3.org > January to March 2017

Re: Fwd: Test Changes Proposal

From: Simon Pieters <simonp@opera.com>
Date: Thu, 16 Feb 2017 13:00:48 +0100
To: "Benjamin Schaaf" <ben.schaaf@gmail.com>
Cc: public-test-infra@w3.org
Message-ID: <op.yvq4zmr9idj3kv@simons-mbp>
On Thu, 16 Feb 2017 01:29:22 +0100, Benjamin Schaaf <ben.schaaf@gmail.com>  

> I'm not really sure how we can tackle the user-agent defined margins
> for rendering. Perhaps an easy way to change it automatically
> throughout the tests would help?
> How do other tests handle this?

Yeah, we could have a single JS file that sets expected horizontal and  
vertical margins, where each vendor can put their own value, and that is  
used in the reference files to create the correct expected rendering...

We have some similar things, like /resources/testharnessreport.js which is  
empty but each vendor can put stuff there for integration of  
testharness.js with some other system.

> As for categorization, I just tested the help link you suggested and
> it doesn't seem to influence test outcome. So producing a report would
> involve parsing all the tests themselves (only a bit more work). But a
> categories.json may still be useful for more high-level reports, eg:
> "This browser passes 97% of the parsing tests for cues, but 0% of the
> parsing tests for regions"

OK. Yeah, maybe it doesn't affect anything in the test runner. Such links  
are used in CSS tests though for e.g. annotation the relevant  
specification with test results. Since WebVTT is generated with bikeshed  
(same as CSS specs), I think it should be possible for WebVTT as well. But  
it doesn't rule out having a JSON file as well. :-)


Simon Pieters
Opera Software
Received on Thursday, 16 February 2017 12:01:26 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 17:34:13 UTC