Re: UAs passing tests if they don't implement a feature

On Jun 24, 2012, at 5:42 AM, Aryeh Gregor wrote:

> On Fri, Jun 22, 2012 at 10:43 AM, fantasai
> <fantasai.lists@inkedblade.net> wrote:
>> Are they easier to review, update, and create and compare derivations
>> also? (As I said, these are not write-only tests.) It's easier to write
>> tests without indentation, too. But harder to read them later.
>> 
>> I have seen some tests by bzbarsky that used the style attribute,
>> and were easy to read and understand. But they had 1-2 declarations
>> on each element, and only a few elements with styles.
>> 
>> In my experience, I've found it easier to work on tests with the
>> style declarations in <style>, where they can be written out with
>> indentation and comments, and can be easily refactored.
> 
> The question is, is this enough justification to require preexisting
> tests to be rewritten?  It's one thing to request particular
> formatting for tests that were written a priori for the CSSWG test
> suite.  It's another thing to refuse to accept existing tests because
> of it.

Ok, I think I'm finally seeing the underlying disconnect here. Please correct me if I'm wrong.

Aryeh, please look here:
http://test.csswg.org/shepherd/search/reviser/AryehGregor/status/rejected/

This is the complete list of all the tests you touched that have been rejected. Note that it's an empty list.

Also check here:
http://test.csswg.org/suites/css3-transforms/nightly-unstable/

This is the current version of the Transforms test suite as created by our nightly builds, note that every one of your submitted tests is present. While this is a nightly build, I can assure you that at no time was any of your tests rejected from the build (if one was missing it's due to a bug in the build code).

None of your tests have _ever_ been rejected for stylistic reasons. Looking through Shepherd's logs, you did receive feedback asking for improvements. I see some feedback that's technical, file name conflicts and the like, and some stylistic feedback. Note that I didn't go though all of your tests so if I missed something significant, please point it out to me.

The stylistic feedback you received was mostly from Dirk, and I read and interpret the bulk of those as him expressing his personal opinion (about that tests should be guide how to write good documents) and a polite request "I would like to ask that…".

Please not that Dirk's opinion here (while not invalid in itself), is _not_ an official requirement of the WG and in no way were your tests rejected for stylistic reasons. 

While there's value in your having modified your tests in response to Dirk's feedback, a perfectly valid response to that would have been "Ok, noted for future tests that I write from scratch" and you could have left it at that (at least as far as I'm concerned).


> 
> Could we create a category of "should" requirements, put all the
> stylistic guidelines there, and say that submitters are asked to
> follow them for new tests but that they aren't grounds for rejecting a
> test in themselves?  An example of a test I wrote before reformatting:
> 
> <!doctype html>
> <link rel=match href=table-4-ref.html>
> <style>table, tbody, tr, td { margin: 0; padding: 0; border-spacing: 0 }</style>
> <!-- preserve-3d is on every intervening element, so this shouldn't vanish -->
> <div style="transform: rotatex(90deg); transform-style: preserve-3d">
>    <table style="transform-style: preserve-3d">
>        <tbody style="transform-style: preserve-3d">
>            <tr style="transform-style: preserve-3d">
>                <td style="transform-style: preserve-3d">
>                    <div style="transform: rotatex(90deg)">Some text</div>
>    </table>
> </div>
> 
> Leave aside the non-stylistic issues (lack of <title> and <meta
> name=assert>, for instance).  Is this really so unreadable that it has
> to be rewritten?  It's twelve lines long.  I really don't think that
> it will take anyone more than twenty seconds to read through it.
> We're not talking about spaghetti code here.  I don't think it makes
> sense to reject this kind of thing outright.

I agree with you completely here (except for the perception that your tests were rejected due to this). 

Purely stylistic issues in tests are in the "nice to have" category. There are expectations for people writing tests from scratch to use best stylistic practices, but for adapting existing tests to our format, our concern is really only getting the required metadata and making sure the test is well formed (and functionally correct, of course).

Received on Sunday, 24 June 2012 14:53:18 UTC