W3C home > Mailing lists > Public > public-css-testsuite@w3.org > September 2010

Re: Tests with difficult/confusing pass condition/wording

From: Gérard Talbot <css21testsuite@gtalbot.org>
Date: Wed, 8 Sep 2010 13:57:33 -0700
Message-ID: <0b22d2b39c44216bbcb8b7793da49ab1.squirrel@cp3.shieldhost.com>
To: "public-css-testsuite@w3.org" <public-css-testsuite@w3.org>
Cc: "Řyvind Stenhaug" <oyvinds@opera.com>
> Hi,
> the following lists a number of tests that were found to be confusing
> regard to determining a test result.
> (Authors mentioned: CSS1 Test Suite Contributors, Elika J. Etemad, Gérard
> Talbot, Hilbrand Edskes, Ian Hickson, James Hopkins, Microsoft,
> Ishida)

Hello Řyvind :)

> ==== CSS1 Test Suite Contributors, Ian Hickson ====
> http://test.csswg.org/suites/css2.1/20100815/html4/c414-flt-ln-002.htm
* The description is too complicated, and it's difficult to tell
> if
> things are really level.

One way to tell is by highlighting the line. And then we realize that
the small coloured boxes ([A][B][C][D]) tops are not perfectly flush
with their correspondent big coloured boxes.

> Maybe this can be split up into multiple
> simpler
> tests?

looks like one but it's not a less complex testcase.

I examined
a bit and I do not see an easy, quick, clear, clean and reliable way to
improve the testcase right now.

Odd.. we discussed this c414-flt-ln-002.htm test before:


but no one seem to have noticed that the small vs big coloured boxes are
not perfectly lined up, flush at their respective top.

> * Says "unless there is not enough room for the big box to fit on the
small box's line after the small box" - but in fact some of the big
> are /left/-floated
> * "big coloured boxes should be level with the top of..." (assuming
> meant that the top border edges of the two boxes should be level) -
> necessarily true. The float's top may not be higher than the top of
> span's linebox, so the TC seems to assume that 1.3em lineheight is
> enough space for a medium border.

That's probably where I would start trying to correct, improve the
testcase. Is there a need to have a wrapping border around each of those
small boxes?

The inline boxes bleed out of the line box; also line-height 1.3 (==
20.8px) means that fractional pixel issues can or will happen.

> http://test.csswg.org/suites/css2.1/20100815/html4/c44-ln-box-003.htm
Wording can be confusing. "just the green outline of an empty box"
> be slightly better

> http://test.csswg.org/suites/css2.1/20100815/html4/c525-font-wt-000.htm
This assumes that there is a "very light" weight available for the
> font or that a tester knows which faces are available and that using
> lightest one is fine

Regarding that c525-font-wt-000.htm test, one idea would be to provide a
link of downloadable fonts (which should be useful or recommendable) for
the test.

Another idea is to make the text font-size much bigger (say 64px) to
give the tester an honest chance to better see and compare. When using
DejaVu Sans font face and with a big font-size, I can see that some
browsers fail that test ... (or that the test is not correct? I haven't
examined the test...it's not a simple one)

I have mentioned
http://test.csswg.org/suites/css2.1/20100815/html4/c527-font-10.htm as a
questionable test in

> ==== Gérard Talbot ====
> http://test.csswg.org/suites/css2.1/20100815/html4/separated-borders-model-007.htm
Uses the term "separated borders", would perhaps be better to describe
expected appearance more directly.

I agree. I need to word the expected results differently... or to redo
the testcase so that the wording of expected results do not pose
understanding or interpretation issues... need to think this over..

> Moreover, this test seems to depend
> on
> the HTML 'rules' attribute mapping to/interacting with CSS rules in a
certain unspecified way (Opera currently applies the horizontal rules
> the <tr>s for instance, haven't investigated what makes Fx/IE fail it;
HTML5 attempts to specify the presentational hints but seems
> unfinished/broken)

This particular matter has been discussed here:

[CSS21] rules="all" attribute specification and border-collapse property

and a "complete set of testcases and agreement on what they should look
like" is available here:


> ==== Gérard Talbot, Hilbrand Edskes ====
> http://test.csswg.org/suites/css2.1/20100815/html4/min-height-106.htm
It's not at all clear from the wording that the green square should be
covered by the scrollbar (which basically means that it will look like
> non-square rectangle)

I think I understand what you mean. Since the horizontal scrollbar
should occupy and use a portion of the height of the green square, then
it no longer makes it a real/perfect green square.

I will modify (after consulting with Hilbrand) the testcase accordingly
so that expected results are more reliable, formal, clear.

Right now, I think we should just overlap a perfect 200px by 200px green
square (with no horizontal scrollbar) over (on top of)

> ==== Gérard Talbot, James Hopkins ====
> http://test.csswg.org/suites/css2.1/20100815/html4/max-height-105.htm
(Probably correct per the current spec, but none of the browsers I
> show a square in this case

Řyvind, I'm not sure I understand you here.

so this might change, see thread starting
> <http://lists.w3.org/Archives/Public/www-style/2010Aug/0541.html>)

I'll get back to you on this one later.

==== Ian Hickson ====
> http://test.csswg.org/suites/css2.1/20100815/html4/background-root-020.htm
There are no "links below". Also, the last two sentences should be
rewritten to not require an understanding of the specification


> http://test.csswg.org/suites/css2.1/20100815/html4/quotes-035.htm
These are missing a proper pass condition


was in Arron's list of testcases


Not sure if quotes-035 was also mentioned elsewhere as well.

It was discussed in


> http://test.csswg.org/suites/css2.1/20100815/html4/tables-001.htm This
test could use a description/an explicit pass condition


> ==== Microsoft ====
> http://test.csswg.org/suites/css2.1/20100815/html4/table-visual-layout-024.htm
This kind of pass condition is confusing in general (when does the
> not pass?).

As worded, the test can not fail. I think the goal is to verify that the
testcase does not make the application crash or create a infinite loop

> In this particular case the "may" is actually also
> incorrect;
> the assert refers to a note, but the note links to a normative
> so
> overflow should be expected. (Of course this assumes that the span's
content is wider than 3em when rendered with the default font,
> a
> safe assumption though using Ahem or non-textual content seems better)

See also
> http://lists.w3.org/Archives/Public/public-css-testsuite/2009Sep/0014.html
- to an observer not familiar with the spec's terminology, it is not
obvious that a thick line counts as a "box"

Agreed. I propose "thick black bar across the page" or "thick black
horizontal line across the page" or "black ribbon across the page"
instead as a replacement.

regards, Gérard
Contributions to the CSS 2.1 test suite:

CSS 2.1 test suite (beta 3; August 15th 2010):

CSS 2.1 test suite contributors:
Received on Wednesday, 8 September 2010 20:58:10 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 21:13:20 UTC