Re: CSS 2.1 test suite feedback: test feedback

> I've been through about 60% of the HTML4 tests in the 20100917 suite,
and have some feedback.
> This message describes issue with individual those tests with issues
that I've covered so far.
> content-type-000 and content-type-001 need to indicate that they are
only valid when tested over http with the appropriate .htaccess
> settings.

Hello Simon,

http://test.csswg.org/suites/css2.1/20101001/html4/content-type-000.htm

http://test.csswg.org/suites/css2.1/20101001/html4/content-type-001.htm

content-type-000 is (RC2) and was (RC1) declaring the http flag; I
deliberately left a comment in the testcase wrt this.

content-type-001 does not rely on any .htaccess setting.

[Addendum: Opera 10.62 passes
http://test.csswg.org/suites/css2.1/20101001/html4/content-type-000.htm
and
http://test.csswg.org/suites/css2.1/20101001/html4/content-type-001.htm
but fails
http://www.gtalbot.org/BrowserBugsSection/css21testsuite/content-type-000.html
and
http://www.gtalbot.org/BrowserBugsSection/css21testsuite/content-type-001.html
so these 2 still requires investigation.
]

> font-size-123 is not self-documenting.

font-size-123 has been corrected:
http://lists.w3.org/Archives/Public/public-css-testsuite/2010Sep/0169.html

> first-letter-nested-006 is not self-documenting.

http://test.csswg.org/suites/css2.1/20101001/html4/first-letter-nested-006.htm

I agree with you and I would add that since the color of first letter is
going to decide in that testcase, then font-size should be bigger. Green
is considered as a dard color; so constrast with black is not as great
as with a light color.

Whenever possible, testcases should comply with
http://www.w3.org/TR/AERT#color-contrast
and with usability and accessibility guidelines wrt contrast/brightness
[Robert Hess MSFT], Jakob Nielsen [useit.com], Aries Arditi [Lighthouse
International], etc.)


> numbers-units-015 is ambiguous: does it mean any two of the boxes?
> units-002, units-003: it would be clearer to say "the same height as"

http://test.csswg.org/suites/css2.1/20101001/html4/units-002.htm

http://test.csswg.org/suites/css2.1/20101001/html4/units-003.htm

units-002.htm:
I disagree. The testcase absolutely needs to check the vertical
alignment of bottom and top of all 3 rectangles: the 2 "É" and the img
will (should!) be all using the ascent space and will (should!) be
"sitting" on the baseline: they will not be using the descent space. 0.8
mult 250px = height of image.

Same thing for units-003.htm and with descent space (below baseline).

Those units-002 and units-003 really must be comparing ascent space and
descent space: so vertical alignment of bottom and top of all 3
rectangles must be compared and are the decisive goals/purposes of those
testcases.

> character-encoding-026: unclear if this needs testing over http.
> c71-fwd-parsing-004: ambiguous whether comment refers to text or
background color.

http://test.csswg.org/suites/css2.1/20100917/html4/c71-fwd-parsing-004.htm

http://test.csswg.org/suites/css2.1/20101001/html4/c71-fwd-parsing-004.htm

Huh... there was/is no comment in that testcase.

  <style type="text/css">
   p.twentythree {text-indent: 0.5in;background:lime}
    color: red
   p.twentyfour {color: red;}
  </style>

color: red is really part of the test itself (on parsing).


> core-syntax-001: green refers to background (needs clarification)
> core-syntax-002: green refers to foreground (needs clarification)

http://test.csswg.org/suites/css2.1/20101001/html4/core-syntax-001.htm
http://test.csswg.org/suites/css2.1/20101001/html4/core-syntax-002.htm

I agree with you.

> matching-brackets-002: failure mode is red on red
> matching-brackets-003: is red background a fail?

http://test.csswg.org/suites/css2.1/20101001/html4/matching-brackets-002.htm

http://test.csswg.org/suites/css2.1/20101001/html4/matching-brackets-003.htm

Maybe both those testcases should state at the beginning that there must
be no red anywhere.

> content-counter-004: 12 squares for reference would be useful,
otherwise
> I have to count them.
> content-counters-004: takes too much effort to score.

http://test.csswg.org/suites/css2.1/20101001/html4/content-counter-004.htm

http://test.csswg.org/suites/css2.1/20101001/html4/content-counters-004.htm

I agree with you: those squares are small and close to each other. So,
the testcases require some time, reading/viewing effort.

> counters-order-001: takes too much effort to score.
> at-charset-001 et al: should clarify what two lines means.
> active-selector-002: confusing
> active-selector-004: ambiguous

http://test.csswg.org/suites/css2.1/20101001/html4/active-selector-004.htm

My neighbour would rightfully, I think, ask me "What is activating a
text?" and/or "How do you activate a text?".

> dom-hover-001: clarify that mouse should not move
> cascade-precedence-001: ambiguous: vertical or horizontal center?

Simon, I have to rethink anyway all those cascade-precedence-00* because
webkit browsers use "text-align: auto" and mozilla browsers use
"-moz-center-or-inherit" (and not text-align).

> first-line-inherit-001 is not self-documenting.

http://test.csswg.org/suites/css2.1/20101001/html4/first-line-inherit-001.htm

I agree with you.


> first-letter-non-punctuation-001 ambiguous

http://test.csswg.org/suites/css2.1/20101001/html4/first-letter-non-punctuation-001.htm

has been fixed (less ambiguous) in RC2 but now, such first character
(&#1470;) is very small: so, not easy to see, verify, confirm that it is
effectively green and not black. Here, I would be for making the
font-size much bigger.

> first-letter-punct-before-015: too long

http://test.csswg.org/suites/css2.1/20101001/html4/first-letter-punct-before-015.htm

is long to load (the tester must have considerable system resources) and
the testcase has 82 validation errors:
http://validator.w3.org/check?uri=http%3A%2F%2Ftest.csswg.org%2Fsuites%2Fcss2.1%2F20101001%2Fhtml4%2Ffirst-letter-punct-before-015.htm&charset=%28detect+automatically%29&doctype=Inline&group=0


> first-letter-punct-before-019: too long

http://test.csswg.org/suites/css2.1/20101001/html4/first-letter-punct-before-019.htm

I agree; the testcase is long to load (the tester must have considerable
system resources). And it has 43 validation markup errors:
http://validator.w3.org/check?uri=http%3A%2F%2Ftest.csswg.org%2Fsuites%2Fcss2.1%2F20101001%2Fhtml4%2Ffirst-letter-punct-before-019.htm&charset=%28detect+automatically%29&doctype=Inline&group=0

> first-letter-punct-before-035: too long

http://test.csswg.org/suites/css2.1/20101001/html4/first-letter-punct-before-035.htm

I agree; the testcase is long to load (the tester must have considerable
system resources). And it has 167 validation markup errors:
http://validator.w3.org/check?uri=http%3A%2F%2Ftest.csswg.org%2Fsuites%2Fcss2.1%2F20101001%2Fhtml4%2Ffirst-letter-punct-before-035.htm&charset=%28detect+automatically%29&doctype=Inline&group=0


> first-letter-punctuation-*: it's hard to see the color of a small
glyph.

I fully agree with you.

> matching-brackets-002: failure mode is red on red, so unreadable.
blocks-017: should say "evenly distributed vertically"

I agree.
It should also be correct to begin with: it should be 184px, not 180px.
The gap is not the same between the top blue border and the first black
stripe and between the 4th black stripe and the bottom blue border.

     height: 180px;
   }
   div > * {
     border: thin solid silver;

to replace with

     height: 184px;
   }
   div > * {
     border: 1px solid silver;

Best would be to get rid of such silver border: it is not necessary to
the testcase and it does not provide anything useful, helpful. Removing
that silver border would simplify the code, the calculations involved.


> table-margin-004: ditto

I agree; it should say "evenly distributed vertically"

> c5519-brdr-r-001: takes too much effort to score.
> border-applies-to: ambiguous ("box" implies solid)

box is not a geometry notion or concept: most people would imagine a 3
dimensional object when thinking/imaginating a "box". Square, rectangle
are geometrical shapes, 2 dimensional objects.

> floats-104: rtakes too much effort to score.

http://test.csswg.org/suites/css2.1/20101001/html4/floats-104.htm

I may agree with you here... but such testcase is not necessarly easy to
improve.


> floats-108: should be a ref test, since it mentions a ref
> floats-126 (and similar): says to resize the window, but not what
should
> happen when you do.

What would you propose to improve such test? I'm not saying that I do
not agree with you here.

> blocks-020: not self-describing, no ref
> blocks-021: ditto
> blocks-022: ditto

http://test.csswg.org/suites/css2.1/20101001/html4/blocks-020.htm
http://test.csswg.org/suites/css2.1/20101001/html4/blocks-021.htm
http://test.csswg.org/suites/css2.1/20101001/html4/blocks-022.htm

Agreed.

> list-style-position-*: takes too much effort to score, needs ref.
before-after-dynamic-restyle-001: uses red in a non-failure case

http://test.csswg.org/suites/css2.1/20101001/html4/before-after-dynamic-restyle-001.htm

I agree.

> content-white-space-003: refers to blue and orange boxes, but boxes
are
> silver.

http://test.csswg.org/suites/css2.1/20101001/html4/content-white-space-003.htm
has NOT been fixed in RC2.

> quotes-035: non-obvious failure
> quotes-036: non-obvious failure

http://test.csswg.org/suites/css2.1/20101001/html4/quotes-035.htm
http://test.csswg.org/suites/css2.1/20101001/html4/quotes-036.htm

Those quotes-035 and quotes-036 testcases have been discussed before. I
do not know what is the status, conclusion, verdict wrt quotes-035 and
quotes-036.

regards, Gérard
-- 
Contributions to the CSS 2.1 test suite:
http://www.gtalbot.org/BrowserBugsSection/css21testsuite/

CSS 2.1 test suite (RC2; October 1st 2010):
http://test.csswg.org/suites/css2.1/20101001/html4/toc.html

CSS 2.1 test suite contributors:
http://test.csswg.org/source/contributors/

Received on Sunday, 3 October 2010 19:20:07 UTC