Re: UAs passing tests if they don't implement a feature

On Tue, 19 Jun 2012 17:36:48 +0200, Aryeh Gregor <ayg@aryeh.name> wrote:

> On the other hand, if the purpose of the tests is at least partly so
> that users can compare standards conformance of different
> implementations, then we do want UAs that don't implement transforms
> at all to fail all these tests.  Otherwise, a partial implementation
> might cause the score on the tests to go *down* relative to no
> implementation, which from this perspective would be bad.

Personally, I don't think there should be a hard requirement that  
absolutely all tests must fail in a UA that doesn't implement. Of course,  
if the percentage of passes is, say, 50%, that's probably a bad sign.

> Relatedly, I would like people's opinions on the following: I do think
> that either way, it's fine for a test to pass in a UA that doesn't
> implement the feature if a conforming UA is supposed to do nothing
> too.

I agree. There should be no need to artificially bolt onto the test  
something that's unrelated to the assert, just to try to ensure a fail.

> E.g., if I have a test that "transform: rotate(10%)" is a parse
> error, then a non-conforming UA *should* pass that test -- the
> property is supposed to not be supported in this case.  I wanted to
> bring this up too to see what everyone else thinks, because Simon
> suggested adding != refs in these cases too.  (It's not clear to me
> what the != ref would even be.)

In this specific case I would probably add a different transform  
declaration in front, e.g.

<style>
div { width: 100px; height: 100px; background: red; }
div + div { transform: translateY(-100px); transform: rotate(10%);  
background: green; }
</style>
<p>There should be a green square below, and no red.</p>
<div></div>
<div></div>

No need for any != ref then. However, there might be other cases where  
there is no natural way of differentiating. And if you really really want  
to test the specific case where the declaration block has only one  
'transform' declaration, I suppose that should be allowed.

-- 
Øyvind Stenhaug
Core Norway, Opera Software ASA

Received on Tuesday, 19 June 2012 16:55:05 UTC