Re: UAs passing tests if they don't implement a feature

On 06/21/2012 05:53 AM, Aryeh Gregor wrote:
> On Wed, Jun 20, 2012 at 8:26 PM, Linss, Peter<peter.linss@hp.com>  wrote:
>>
>> I'm going to correct my previous statement about the != ref. In the rotate % case I think the != ref _should_ be done via rotation, as should your != ref for the example you gave.
>>
>> Concretely, for transform-background-006, there should be a != reference which is the same as the == reference except put the 'transform: rotate(90deg);' on the html element along with the background (and remove the transform-origin).
>>
>> This would cause UAs that don't implement transforms to render the == and != references identically and therefore fail the test.
>
> That's possible, sure.  Really, though, the != ref in this case is red
> herring -- it doesn't have anything to do with what's being tested, if
> you think about it.  I could also add a != ref with 'transform:
> scale(3.5) translateX(-21em)', and it would have exactly the same
> effect.  All that tests is that transforms do something.  (In general,
> all notrefs test are "this change does *something*", without testing
> what.)

See my response here:
   http://lists.w3.org/Archives/Public/public-css-testsuite/2012Jun/0021.html
IIRC, pretty much all of hixie's tests (and, I hope, mine as well)
that test for a parse error ensure that there is a valid declaration
that takes effect if the parse error is correctly ignored, because
that's a stronger test for correct error handling.

> This is the kind of thing that strongly discourages contributions to
> the CSS testsuite: requiring test submitters to repeatedly rewrite
> submitted tests that are technically correct and serve their purpose
> perfectly well, based on long lists of written and unwritten rules
> that are not disclosed in advance of the test submissions and that the
> submitter may or may not think make sense.  That kind of policy works
> when submitters are forced to work with the CSSWG, but not otherwise.

The not refs suggestion wasn't part of the written rules when you
wrote the tests because Simon brought it up less than a week ago.

Btw, the CSSWG tests aren't write-once, dump it in the regression suite
tests, never look at it again. People who didn't write the test need to
read and understand their code later. Improvements are made over time.
That's part of the process.

> But for now, I have tests to submit.  And adding some != refs here is
> a lot less onerous than rewriting all my style attributes as<style>
> tags, which I'm also spending hours doing because the CSSWG apparently
> feels my time is better spent doing that than actually writing
> substantive new tests (or other useful work).

I don't have a strong objection to accepting existing tests that use
the style attribute, as long as they are well-written and the code
is organized in such a way as to be easily understandable. There are
certainly advantages to writing the code in the style attribute; but
there are also disadvantages, such as when declarations are shared
across elements, or when the style block is long. In many cases,
organizing the code for better readability will require factoring out
most of the styles into <style>. Also, although I'm sure you can find
exceptions, in general I think it's easier to refactor and improve
the usability of tests when they are using <style> elements.

I will point out that this recommendation has been in the format
documentation for many years at this point, so your aforementioned
complaint about changing rules does not apply.

~fantasai

Received on Friday, 22 June 2012 06:13:09 UTC