Re: Better Test Case Coding for Reviewers

> On 05 Jul 2015, at 22:42, Gérard Talbot <css21testsuite@gtalbot.org> wrote:
> 
> Le 2015-07-05 12:31, fantasai a écrit :
>> [CCing m.d.platform, since it might be helpful for layout tests there, too.]
>> I've been reviewing some tests lately, and there are three things that
>> would help a lot in a correct and efficient review.
>> #1: Good indentation.
>>  The contents of each block should be indented. It's much harder to
>>  read the style sheet if this is not true. I've seen quite a few tests
>>  that have this problem. :(
> 
> 
> I am for this. We need to give an example of recommended indentation:
> 
>  <style>
>  p
>    {
>      margin: 1em 0em;
>    }
>  </style>
> 
> and the documentation itself should also start giving the example, practicing its very own recommendations.

I think asking for things to be indented, and for a consistent indentation style to be used within one test case is useful, reasonable, and should not cause any controversy.

I don't think picking one particular indentation style has nearly as much value, and it is likely to be much more contentious which indentation style we should recommend.



>> #2: Separating framework from variation.
>>  Most tests have code that sets up a framework to reveal the results
>>  of a test, and then code that sets up the specific situation being
>>  tested. The framework does not vary from test to test within a set,
>>  but the test declarations do.
> 
> I wish we had this kind of email discussion years ago. Before CSS2.1 test suite even started. Especially before thousands of tests get created.

I don't think I've been systematic about this myself, but it is a good suggestion.

>> #3: Declaration of intent
>>  If the test writer tells me what they're trying to test, specifically,
>>  in this test, rather than having me guess from the source code, I'm
>>  much more likely to detect when they're failing at their job, either
>>  by flat out not triggering the right situation at all, or by not
>>  checking some modes of failure. (I see both of these happen often;
>>  it is not a theoretical problem.)
> 
> Both the <title> and the meta assert text should be doing all that.
> 
> Sometimes, the multiple directives, requirements can overconstrain the recommendation itself: eg the title should be short and descriptive... but sometimes, this is just impossible to have both.

I usually find titles much harder to write than assertions, for this reason.

For assertions, I think reminding people that they should write one, and giving a bunch of examples should be enough. The goal is clear as it is.

For titles, we should probably give it some more thought, and clarify the expectation.

 - Florian

Received on Monday, 6 July 2015 09:41:51 UTC