Re: Tests with difficult/confusing pass condition/wording: c414-flt-ln-002 , table-visual-layout-024

> On 08/09/10 21:57, "Gérard Talbot" wrote:
>>> ==== CSS1 Test Suite Contributors, Ian Hickson ====
>>> http://test.csswg.org/suites/css2.1/20100815/html4/c414-flt-ln-002.htm
>> * The description is too complicated, and it's difficult to tell
>> easily
>>> if
>>> things are really level.
>>
>> One way to tell is by highlighting the line. And then we realize that
>> the small coloured boxes ([A][B][C][D]) tops are not perfectly flush
>> with their correspondent big coloured boxes.
>>
>>> Maybe this can be split up into multiple
>>> simpler
>>> tests?
>>
>>
>> http://test.csswg.org/suites/css2.1/20100815/html4/c414-flt-ln-003.htm
>> looks like one but it's not a less complex testcase.
>>
>> I examined
>> http://test.csswg.org/suites/css2.1/20100815/html4/c414-flt-ln-002.htm
>> a bit and I do not see an easy, quick, clear, clean and reliable way
>> to
>> improve the testcase right now.
>>
>> Odd.. we discussed this c414-flt-ln-002.htm test before:
>>
>> http://lists.w3.org/Archives/Public/public-css-testsuite/2010Jul/0044.html
>>
>> but no one seem to have noticed that the small vs big coloured boxes
>> are
>> not perfectly lined up, flush at their respective top.
>
> Highlighting the line is only helpful if they do actually render on the
> same line: I've almost never seen them actually all get rendered on the
> same line


With Opera 10.61, in this c414-flt-ln-002.htm test, they are almost
never rendered on the same line. I am not sure why.

>, and I've seen it go both ways, having multiple large boxes on
> a single line as well as not having the small box on the same line as
> the large one.

I think this can be worked out:
- reducing number of large boxes
- making large boxes narrower a bit
- making large boxes spread vertically, not too close from each other in
source code; there is a lot of flexibility here..
- reducing the number of "filler text" between large and small boxes
- etc.

I am fairly confident that those c414-flt-ln-00x.htm testcases can be
improved.


> The main difficulty with that test is the low likelihood you actually
> get all the boxes on the same line.
>
> Looking at the test more closely now, I'm not sure what the point of all
> those blocks is. I don't see why we need three float: left boxes.

I agree.


> c414-flt-ln-003.htm appears to be testing the same thing, and marginally
> simplier insofar as there are only two boxes. It'd seem better yet to
> have each of those in a different test.
>
>>> ==== Microsoft ====
>>> http://test.csswg.org/suites/css2.1/20100815/html4/table-visual-layout-024.htm
>> This kind of pass condition is confusing in general (when does the
>> test
>>> not pass?).
>>
>> As worded, the test can not fail. I think the goal is to verify that
>> the
>> testcase does not make the application crash or create a infinite loop
>> (hanging).
>
> If that's the case, then it shouldn't have a pass condition dependent
> upon rendering (which, as it stands, is nonsensical). It makes no sense
> to have a testcase that cannot fail. ("This testcase must not crash the
> UA" is a better pass condition than what there is now).


I'd prefer Arron to explain table-visual-layout-024.htm as it is.

There may be a few testcases in the test suite
(eg
http://test.csswg.org/suites/css2.1/20100815/html4/min-width-percentage-003.htm
)
where the rendered layout is undefined and if extensive coverage of the
spec by the test suite (testability of statements found in the spec,
including those with "... is undefined") is a goal, then there is some
reason for such testcases.

Possible more info in this thread:
http://lists.w3.org/Archives/Public/public-css-testsuite/2010Jan/0000.html

regards, Gérard
-- 
Contributions to the CSS 2.1 test suite:
http://www.gtalbot.org/BrowserBugsSection/css21testsuite/

CSS 2.1 test suite (beta 3; August 15th 2010):
http://test.csswg.org/suites/css2.1/20100815/html4/toc.html

CSS 2.1 test suite contributors:
http://test.csswg.org/source/contributors/

Received on Thursday, 9 September 2010 21:05:33 UTC