Re: Tests with difficult/confusing pass condition/wording

On 08/09/10 21:57, "Gérard Talbot" wrote:
>> ==== CSS1 Test Suite Contributors, Ian Hickson ====
>> http://test.csswg.org/suites/css2.1/20100815/html4/c414-flt-ln-002.htm
> * The description is too complicated, and it's difficult to tell
> easily
>> if
>> things are really level.
>
> One way to tell is by highlighting the line. And then we realize that
> the small coloured boxes ([A][B][C][D]) tops are not perfectly flush
> with their correspondent big coloured boxes.
>
>> Maybe this can be split up into multiple
>> simpler
>> tests?
>
>
> http://test.csswg.org/suites/css2.1/20100815/html4/c414-flt-ln-003.htm
> looks like one but it's not a less complex testcase.
>
> I examined
> http://test.csswg.org/suites/css2.1/20100815/html4/c414-flt-ln-002.htm
> a bit and I do not see an easy, quick, clear, clean and reliable way to
> improve the testcase right now.
>
> Odd.. we discussed this c414-flt-ln-002.htm test before:
>
> http://lists.w3.org/Archives/Public/public-css-testsuite/2010Jul/0044.html
>
> but no one seem to have noticed that the small vs big coloured boxes are
> not perfectly lined up, flush at their respective top.

Highlighting the line is only helpful if they do actually render on the 
same line: I've almost never seen them actually all get rendered on the 
same line, and I've seen it go both ways, having multiple large boxes on 
a single line as well as not having the small box on the same line as 
the large one.

The main difficulty with that test is the low likelihood you actually 
get all the boxes on the same line.

Looking at the test more closely now, I'm not sure what the point of all 
those blocks is. I don't see why we need three float: left boxes.

c414-flt-ln-003.htm appears to be testing the same thing, and marginally 
simplier insofar as there are only two boxes. It'd seem better yet to 
have each of those in a different test.

>> ==== Microsoft ====
>> http://test.csswg.org/suites/css2.1/20100815/html4/table-visual-layout-024.htm
> This kind of pass condition is confusing in general (when does the
> test
>> not pass?).
>
> As worded, the test can not fail. I think the goal is to verify that the
> testcase does not make the application crash or create a infinite loop
> (hanging).

If that's the case, then it shouldn't have a pass condition dependent 
upon rendering (which, as it stands, is nonsensical). It makes no sense 
to have a testcase that cannot fail. ("This testcase must not crash the 
UA" is a better pass condition than what there is now).

> See also
>> http://lists.w3.org/Archives/Public/public-css-testsuite/2009Sep/0014.html
> - to an observer not familiar with the spec's terminology, it is not
> obvious that a thick line counts as a "box"
>
> Agreed. I propose "thick black bar across the page" or "thick black
> horizontal line across the page" or "black ribbon across the page"
> instead as a replacement.

"thick black horizontal line" is probably best. ("across the page" is 
mostly redundant with it explicitly being horizontal, and can be 
confusing, as it's not entirely across the page due the margins at 
either side.)

-- 
Geoffrey Sneddon — Opera Software
<http://gsnedders.com>
<http://opera.com>

Received on Thursday, 9 September 2010 13:13:17 UTC