RE: Feature Rings, Incomplete Rings, and Compliance Grading

> -----Original Message-----
> From: Robin Berjon [mailto:robin@berjon.com]
> Sent: Friday, May 04, 2012 3:20 AM
> To: Jet Villegas
> Cc: public-coremob@w3.org
> Subject: Re: Feature Rings, Incomplete Rings, and Compliance Grading
> 
> On May 4, 2012, at 03:25 , Jet Villegas wrote:
> > I'm suggesting that the current system is pretty easy to game, an example
> from the CSS text tests:
> >
> >  assert( H.test.cssProp( elem, "wordBreak" ), "wordBreak supported" );
> >
> > We can "fix" Firefox to pass that test and go 100% green on ring 0 without
> having any word breaking at all. The expensive part of wordBreak isn't
> exposing the "wordBreak" string in the JS API's (yet that's all you need to
> pass!) It's premature to assert that you're testing for wordBreak with the
> tests as coded. I would leave it off any rings until it actually does something.
> 
> A number of the tests in the suite only do feature detection rather than
> testing. That's a bug. We have a bunch of tests that were submitted from
> Ringmark to review in this group, and we're putting together a checklist of
> stuff to watch out for. Doing only feature-detection is already on it.
> 

Test approval is beginning next week.  Once that is under way, it would be great to get these into GitHub issues so that we  can build them out.  Even better, it would be awesome if you would contribute tests that you felt were missing.  The goal of Ringmark is to be a comprehensive set of tests that include common issues that developers hit on a daily basis-- not merely surface-area testing.

We have limited resources to build out tests though, which is why it's important that we get as many eyeballs on this as possible-- so we can fix the holes together.

Received on Friday, 4 May 2012 17:48:53 UTC