Re: Exit criteria Re: [selectors-api] Transitioning to CR

I'm about to issue a CfC on publishing Selectors as a CR, independent of  
getting the test suite done. Because it has taken a long time not to get  
it done, and the result is no CR.

We will need to agree on a Test Suite, and on exit criteria. So this  
message is to see if there is any disagreement.

A possible question is whether it counts to have a JS implementation or  
similar. I think it would be reasonable, so long as it is a completely  
independent implementation, although I would rather have it *in addition  
to* native implementations in shipping products.

On Wed, 24 Jun 2009 15:56:11 +0200, Lachlan Hunt  
<lachlan.hunt@lachy.id.au> wrote:

> Charles McCathieNevile wrote:
>> Actually, based on feedback on the list (thanks Maciej and Robin), and
>> talking to Lachy, we are thinking that we should seperate out the tests
>> that *require* CSS 3 selectors, to make the test suite check
>> implementation of the API, and then require at least two 100% complete
>> and completely interoperable implementations.
>>
>> I believe Lachy will be following up on this about now - both for the
>> list and the test suite.
>
> Here is the revised proposal for the exit criteria.
>
> * Tested implementations are required to have support for:
>    - Selectors API
>    - Selectors defined in CSS 2.1.
>    - HTML
>
> * Tested implementaions may optionally support:
>    - Selectors introduced in Selectors Level 3
>    - XHTML
>    - SVG
>
> At least two implementations must pass 100% of the baseline testsuite  
> and should pass additional tests, dependent on the following conditions:
>
> * The baseline testsuite comprises tests that check for conformance to
>    all requirements in the API using only HTML and Selectors defined in
>    CSS 2.1.
>
> * Tests using Selectors introduced in Selectors Level 3, or XHTML+SVG,
>    are considered to be additional tests.

I wonder if we need to make these additional tests rather than baseline,  
for the purposes of demonstrating that browsers get this spec right.

> * An additional test may be marked as N/A for an implementation if:
>    - The test uses a selector that the implementation does not support
>    - The test uses XHTML+SVG that the implementation does not support
>
> * Implementations are not required to pass all additional tests,
>    however no failures must be caused by an incorrect implementation of
>    the API itself. Failures of additional tests caused only by an
>    incorrect implementation of Selectors do not count.
>
>
> This implies that the testsuite should be split into several files:

I think that at most we should be designating tests as baseline or  
additional, rather than trying to classify the various kinds of additional  
files.

> 1. Baseline containing tests using only HTML and CSS 2.1
> 2. Additional tests using XHTML+SVG and CSS2.1 (equivalent to the
>     previous test, but with the addition of SVG-related tests)

Maybe, maybe not, as per above.

> 3. Additional tests using HTML and Selectors 3
> 4. Additional tests using XHTML and Selectors 3

cheers

Chaals

-- 
Charles McCathieNevile  Opera Software, Standards Group
     je parle français -- hablo español -- jeg lærer norsk
http://my.opera.com/chaals       Try Opera: http://www.opera.com

Received on Wednesday, 18 November 2009 23:50:34 UTC