Re: Evolving ringmark

On 06/22/2012 05:03 PM, Thomas, Gavin, Vodafone Group wrote:

> Currently ringmark [2] declares to make 382 tests covering 78
> features. There exists well known tools such as html5test [3] and
> css3test [4] and others including W3C tests and browser vendor tests.
> I hope we can leverage these.

The name "test" in the title of those is rather misleading. The vast 
majority of their "tests" are not checking any sort of conformance but 
simple feature detection (clearly 2-5 tests per feature, and no 
possibility to check rendering or anything else that can't be covered by 
DOM/javascript isn't going to be any sort of conformance test).

It is very unclear what the advantage of such a test is over a manually 
curated list of which features a browser supports; the latter has the 
benefit that it can account for QoI issues.

> At Vodafone, we are open to adopting a shared test tool and focus
> some resources to contribute to make something better for all. I
> think others have a similar view. However so far it has been
> difficult to 1) understand the scope of ringmark and 2) identify
> where focussed contributions are required.

The area where contributions are most valuable is actually writing 
tests; I would guesstimate that there are around three orders of 
magnitude more useful work to be done making tests than making tooling 
around tests; this doesn't mean that tooling isn't important but that we 
are chronically short of high quality cross-browser tests for many areas 
of the platform.

If you are going to write tests, please submit them to the relevant W3C 
working group. There is no advantage to anyone of developing an 
independent silo of tests here.

Received on Monday, 25 June 2012 08:50:54 UTC