W3C home > Mailing lists > Public > public-css-testsuite@w3.org > September 2009

General and specific comments, feedback on Microsoft submitted tests

From: Gérard Talbot <css21testsuite@gtalbot.org>
Date: Mon, 28 Sep 2009 17:47:20 -0400 (EDT)
Message-ID: <1662.173.179.227.155.1254174440.squirrel@69.59.28.71>
To: public-css-testsuite@w3.org
Hello all,

Several of Microsoft submitted tests are not optimal or it seems to me
that they could be improved in several areas. The description of expected
results is sometimes too unspecific or not specific, not precise enough,
not measurable, not constraining enough.

Comparing 2 small squares
=========================
Comparing 2 small squares of approx. 15 pixels can be difficult,
precisely if the pass condition is that both squares should be using the
same height and/or the same width. Who would be able to notice a 1 or 2
pixels difference between 2 squares of approx. 15px each?

Filled box or just black-bordered box?
======================================
Sometimes, one of the Microsoft test mentions a black box but in reality,
only the border of the box is black; it's not a filled box, fully painted
black box. And sometimes, another Microsoft test mentions a box but it's a
filled, fully painted black box.

More relevant shape description
===============================
A "box" is somewhat a general shape description. I wish "box" would
be avoided for CSS 2.1 test suite. Speaking of a rectangle or a square or
a line or a stripe may be more precise, more useful, more relevant. Better
words describing better the shapes in expected results.
Same thing with "block" (Ian Hickson): as a shape, it's not descriptive
enough.

Better id semantic
==================
The coding of id attribute is also not best, not optimal in my opinion.

<div id="div1"> means nothing in the absolute, otherwise little in
relativity (it just means the first encountered div in code order).

E.g.:
<div id="grandparent-container"> is more useful, relevant, meaningful.
<div id="parent-blue-container"> is helpful.
<span id="floated-left"> is helpful.
<div id="control">, <div id="test"> helps understand the building logic of
the test.

The CSS 2.1 test suite can (auxilliary/corrolarly speaking) help web
authors into understanding how CSS properties work if they can study the
tests source code.

Specific comments regarding specific Microsoft submitted tests
**************************************************************

Those comments are preliminary and covers randomly checked tests in
"chapter 10".

http://test.csswg.org/source/contributors/microsoft/submitted/Chapter_10/Containing-Block-010.xht

Suggestion:
Test passes if a blue square is in the upper-right corner of a bigger
black square.

Comment: The black square should be a square filled with black, not just a
black border. Otherwise, the description of expected results should
indicate that the square has black borders and is not filled with black.

There are lots of testcases like that one.

--------------

http://test.csswg.org/source/contributors/microsoft/submitted/Chapter_10/Max-Width-Percentage-001.xht

Suggestion: Test passes if the blue and orange squares below are the same
width.

--------------

http://test.csswg.org/source/contributors/microsoft/submitted/Chapter_10/Max-Width-Percentage-003.xht

Suggestion:
How about using lime instead of green since the "Filler Text" words are
going to be written in that small rectangle. Sufficient color brightness
and color contrast is very important according to by Lighthouse
International (Effective Color Contrast), J. Nielsen (Top Ten Web Design
Mistakes of 2005: 1. Legibility Problems), WAI W3C (Techniques For
Accessibility Evaluation And Repair Tools) and Robert Hess, MSDN

"Contrast is the perceived difference between two adjacent colors. (...)
Contrast is extremely important in Web design."
Wheel of Color: Pump up the Contrast, Robert Hess, MSDN

-------------


http://test.csswg.org/source/contributors/microsoft/submitted/Chapter_10/absolute-non-replaced-height-001.xht

Suggestion:
Test passes if a blue square is in the upper-left corner of the large
black rectangle.

Comment: the black rectangle should be a filled with black, not just
having a black border.

-------------

http://test.csswg.org/source/contributors/microsoft/submitted/Chapter_10/absolute-non-replaced-max-height-002.xht

Suggestion:
Test passes if a blue rectangle is in the upper-left corner of a bigger
black square and there is no red visible on the page.

Comment: the black rectangle should be a filled with black, not just
having a black border. Otherwise the test should mention non-filled
rectangle, just a thick black border.

----------------

Suddenly, in this test

http://test.csswg.org/source/contributors/microsoft/submitted/Chapter_10/block-non-replaced-height-008.xht

the word square is mentioned and not box!

----------------

http://test.csswg.org/source/contributors/microsoft/submitted/Chapter_10/float-replaced-width-012.xht

Comment and suggestion:
Such test as is should be rejected, otherwise should be redone. Too small,
too tiny, microscopic. I should not have to magnify my browser view for to
verify if a browser passed or failed this test.


----------

http://test.csswg.org/source/contributors/microsoft/submitted/Chapter_10/height-006.xht

Suggestion:
Test passes if the two black squares below have the same height.

-----------

http://test.csswg.org/source/contributors/microsoft/submitted/Chapter_10/height-025.xht

and

http://test.csswg.org/source/contributors/microsoft/submitted/Chapter_10/height-080.xht

and

http://test.csswg.org/source/contributors/microsoft/submitted/Chapter_10/max-height-025.xht

Suggestion:
Test passes if there is a black line across the page below.

---------

http://test.csswg.org/source/contributors/microsoft/submitted/Chapter_10/height-058.xht

and

http://test.csswg.org/source/contributors/microsoft/submitted/Chapter_10/max-height-058.xht

Suggestion:
Test passes if there is a wide rectangle across the page below.

---------

http://test.csswg.org/source/contributors/microsoft/submitted/Chapter_10/line-height-001.xht
  and line-height-002.xht   line-height-003.xht  line-height-004.xht    
line-height-005.xht   line-height-057.xht

Comment: I wonder if the visual comparison of the height of small squares
couldn't be made easier. What if there is a height difference of 1 or 2
pixels here? It wouldn't be easily and quickly differentiable,
distinguishible. Those tests, as they are, are not ideal.

---------

http://test.csswg.org/source/contributors/microsoft/submitted/Chapter_10/vertical-align-007.xht

Comment: "Test passes if there is at least one black box and there is no
red visible on the page."  I wish the expected results would be more
constraining, more measurable, less flexible, arbitrary. There could be
10, 50 boxes appearing in that test, you know.


About line-height, vertical-align
=================================

Personally, I think that all of the tests regarding vertical-align should
be using a specially designed image for that purpose. So that such
testcases may also be useful for understanding how the vertical-align
actually work. The test suite could be a truly terrific tool for
learning... not just assessing which browser (and browser versions)
support, complies fully with the spec.
The nr 1 reason why previous browser versions failed at supporting
correctly line-height, vertical-align is that tests (CSS 1 test suite, E.
Meyer draft CSS 2 test suite) were not precise, pass vs fail condition
were not easy to figure out.

regards, Gérard
Received on Monday, 28 September 2009 21:48:11 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Monday, 20 September 2010 17:51:58 GMT