W3C home > Mailing lists > Public > public-coremob@w3.org > May 2012

Re: Feature Rings, Incomplete Rings, and Compliance Grading

From: Tobie Langel <tobie@fb.com>
Date: Fri, 4 May 2012 06:41:27 +0000
To: Jet Villegas <jet@mozilla.com>
CC: "robin@berjon.com" <robin@berjon.com>, "public-coremob@w3.org" <public-coremob@w3.org>, "jason@cloudfour.com" <jason@cloudfour.com>, jeanfrancois moy <jeanfrancois.moy@orange.com>, Charles McCathieNevile <chaals@opera.com>
Message-ID: <CBC94635.81169%tobie@fb.com>


On 5/4/12 3:25 AM, "Jet Villegas" <jet@mozilla.com> wrote:

>I'm suggesting that the current system is pretty easy to game, an example
>from the CSS text tests:
>
>  assert( H.test.cssProp( elem, "wordBreak" ), "wordBreak supported" );
>
>We can "fix" Firefox to pass that test and go 100% green on ring 0
>without having any word breaking at all. The expensive part of wordBreak
>isn't exposing the "wordBreak" string in the JS API's (yet that's all you
>need to pass!) It's premature to assert that you're testing for wordBreak
>with the tests as coded. I would leave it off any rings until it actually
>does something.

Let's not mixup quality and coverage extent of the test suite with spec
content.

>That's my concern about the rings as they are pass/fail. In my example,
>the expensive part of getting a "100% correct" wordBreak implementation
>is probably breaking words with complex ligatures on right-to-left Arabic
>text. Will ringmark really "fail" a browser with a zero score for
>wordBreak on such a test?  Let's give them grades instead

No. Let's write more better tests.

--tobie
Received on Friday, 4 May 2012 06:42:33 UTC

This archive was generated by hypermail 2.3.1 : Friday, 19 April 2013 17:36:46 UTC