RC1 : 1st preliminary review report: list of 69 incorrect testcases

Hello all,

This report is mostly for Fantasai, Arron Eicholz and Boris Zbarsky.

Here's a list of 69 testcases which I believe are incorrect and/or
requiring code redesign of some sort (with at least 18 of them
rejectable, non-improvable). These do not involve tiny or trivial
errors.
I tried to gather the most serious issues occuring in testcases.


1-
http://test.csswg.org/suites/css2.1/20100917/html4/cascade-006.htm
http://test.csswg.org/source/contributors/microsoft/submitted/Chapter_6/cascade-006.htm
Why?
http://lists.w3.org/Archives/Public/public-css-testsuite/2010Jul/0035.html

Need code adjustment and text assert adjustment.

--------------------

2- Margin-bottom does *not* apply to table-cells.

http://test.csswg.org/suites/css2.1/20100917/html4/margin-bottom-applies-to-007.htm

eg
<meta name="assert" content="The 'margin-bottom' property applies to
elements with a display of table-cell.">


to change to

<meta name="assert" content="The 'margin-bottom' property does not to
elements with a display of table-cell.">

--------------------

3- Questionable testcases on font and vertical-align

Some of them may be rejectable and non-improvable as initially coded and
designed.

http://test.csswg.org/suites/css2.1/20100917/html4/font-size-061.htm
http://test.csswg.org/suites/css2.1/20100917/html4/font-size-062.htm
http://test.csswg.org/suites/css2.1/20100917/html4/c527-font-000.htm
http://test.csswg.org/suites/css2.1/20100917/html4/c527-font-002.htm
http://test.csswg.org/suites/css2.1/20100917/html4/c527-font-003.htm
http://test.csswg.org/suites/css2.1/20100917/html4/c527-font-004.htm
http://test.csswg.org/suites/css2.1/20100917/html4/c527-font-005.htm
http://test.csswg.org/suites/css2.1/20100917/html4/c527-font-006.htm
http://test.csswg.org/suites/css2.1/20100917/html4/c527-font-007.htm
http://test.csswg.org/suites/css2.1/20100917/html4/c527-font-008.htm
http://test.csswg.org/suites/css2.1/20100917/html4/c527-font-009.htm
http://test.csswg.org/suites/css2.1/20100917/html4/c527-font-10.htm
http://test.csswg.org/suites/css2.1/20100917/html4/vertical-align-115.htm
http://test.csswg.org/suites/css2.1/20100917/html4/vertical-align-116.htm
http://test.csswg.org/suites/css2.1/20100917/html4/c544-valgn-002.htm
http://test.csswg.org/suites/css2.1/20100917/html4/c544-valgn-003.htm
http://test.csswg.org/suites/css2.1/20100917/html4/c544-valgn-004.htm

Reasons:
http://lists.w3.org/Archives/Public/public-css-testsuite/2010Aug/0000.html
http://lists.w3.org/Archives/Public/public-css-testsuite/2010Aug/0002.html

Some can/could be rehabilitated; others would need rewrite or entirely
redesigned.

--------------------

4-
http://test.csswg.org/suites/css2.1/20100917/html4/padding-bottom-036.htm
can create a Fractional pixel problem in browsers and does create a
Fractional pixel problem in Konqueror 4.4.5

--------------------

5- Padding applies to table elements and inline-table elements

http://test.csswg.org/suites/css2.1/20100917/html4/padding-top-applies-to-013.htm
http://test.csswg.org/suites/css2.1/20100917/html4/padding-top-applies-to-014.htm
http://test.csswg.org/suites/css2.1/20100917/html4/padding-right-applies-to-013.htm
http://test.csswg.org/suites/css2.1/20100917/html4/padding-right-applies-to-014.htm

need to be corrected.

--------------------

6- Fractional pixel problem.

http://test.csswg.org/suites/css2.1/20100917/html4/border-bottom-width-036.htm

http://test.csswg.org/suites/css2.1/20100917/html4/border-bottom-width-047.htm

http://test.csswg.org/suites/css2.1/20100917/html4/border-left-width-047.htm

http://test.csswg.org/suites/css2.1/20100917/html4/border-right-width-047.htm

http://test.csswg.org/suites/css2.1/20100917/html4/border-top-width-036.htm

http://test.csswg.org/suites/css2.1/20100917/html4/padding-top-014.htm

http://test.csswg.org/suites/css2.1/20100917/html4/padding-top-050.htm

http://test.csswg.org/suites/css2.1/20100917/html4/padding-top-051.htm
(this one affects Opera 10.62 and Konqueror 4.4.5)

http://test.csswg.org/suites/css2.1/20100917/html4/width-091.htm

http://test.csswg.org/suites/css2.1/20100917/html4/max-width-091.htm

http://test.csswg.org/suites/css2.1/20100917/html4/min-width-091.htm

http://test.csswg.org/suites/css2.1/20100917/html4/height-091.htm

http://test.csswg.org/suites/css2.1/20100917/html4/max-height-091.htm

Fractional pixel problem.

All testcases with a specified length of 1pt, 1mm, *ex, 1% as the
defined target of the testcase should be rechecked to prevent fractional
pixel problem.

3pt == 4px; so best is to use a factor of 3 for pt unit
5ex (ahem) = 4em; so best is to use a factor of 5 for ex unit; ideal is
20px (or 30px or 40px, etc) for parent font size.
mm and cm are more complex to deal with.
%: 25% of 1in == 24px.

--------------------

7-
http://test.csswg.org/suites/css2.1/20100917/html4/display-applies-to-002.htm
needs updating. I have proposed a replacement:
http://www.gtalbot.org/BrowserBugsSection/css21testsuite/display-applies-to-002.htm

--------------------

8-
http://test.csswg.org/suites/css2.1/20100917/xhtml1/html-attribute-019.xht
and
http://test.csswg.org/suites/css2.1/20100917/html4/html-attribute-019.htm

I have proposed replacements:
http://www.gtalbot.org/BrowserBugsSection/css21testsuite/html-attribute-019.htm
and
http://www.gtalbot.org/BrowserBugsSection/css21testsuite/html-attribute-019.xht


--------------------

9- Width may be too tight.

http://test.csswg.org/suites/css2.1/20100917/html4/floats-006.htm

http://test.csswg.org/suites/css2.1/20100917/html4/floats-009.htm

"Filler text" occupies 86px in many fonts. If the font creates a > 96px
Filler Text, like in Courier New (== 110px), then the testcase fails.

#div1
            {
                height: 2in;
                width: 4in; /* proposed adjustment */
            }


--------------------

10- Space available to view bullet list-marker and make it part of pass
condition.

http://test.csswg.org/suites/css2.1/20100917/html4/background-color-applies-to-010.htm

I propose to add a margin-left: 2em; and then to update the expected
result accordingly to mention that a bullet list-marker should be
rendered on the left of the black square.

<style type="text/css">
        div
        {
          background-color: black;
          display: list-item;
          height: 1in;
          margin-left: 2em;
          width: 1in;
        }
</style>

The same kind of code adjustment should be done for other applies-to
'display: list-item' kind of testcases:

http://test.csswg.org/suites/css2.1/20100917/html4/border-collapse-applies-to-003.htm
http://test.csswg.org/suites/css2.1/20100917/html4/padding-top-applies-to-010.htm
and possibly a few other *-applies-to-010 testcases.

--------------------

11- Rejectable testcases because undefined in the section 10.3.x of the
spec. and I do not see how such testcases could be rescuable,
rehabilitable

http://test.csswg.org/suites/css2.1/20100917/html4/inline-replaced-width-007.htm

http://test.csswg.org/suites/css2.1/20100917/html4/block-replaced-width-007.htm

http://test.csswg.org/suites/css2.1/20100917/html4/float-replaced-width-012.htm

http://test.csswg.org/suites/css2.1/20100917/html4/absolute-replaced-width-007.htm

http://test.csswg.org/suites/css2.1/20100917/html4/absolute-replaced-width-014.htm

http://test.csswg.org/suites/css2.1/20100917/html4/absolute-replaced-width-021.htm

http://test.csswg.org/suites/css2.1/20100917/html4/absolute-replaced-width-028.htm

http://test.csswg.org/suites/css2.1/20100917/html4/absolute-replaced-width-035.htm

http://test.csswg.org/suites/css2.1/20100917/html4/absolute-replaced-width-042.htm

http://test.csswg.org/suites/css2.1/20100917/html4/absolute-replaced-width-049.htm

http://test.csswg.org/suites/css2.1/20100917/html4/absolute-replaced-width-056.htm

http://test.csswg.org/suites/css2.1/20100917/html4/absolute-replaced-width-063.htm

http://test.csswg.org/suites/css2.1/20100917/html4/absolute-replaced-width-070.htm

http://test.csswg.org/suites/css2.1/20100917/html4/absolute-replaced-width-077.htm

http://test.csswg.org/suites/css2.1/20100917/html4/inline-block-replaced-width-007.htm


--------------------

12- Technical expected results

http://test.csswg.org/suites/css2.1/20100917/html4/outline-width-093.htm
My neighbour would ask "what's a medium outline? Medium is comparable to
what exactly?"

http://test.csswg.org/suites/css2.1/20100917/html4/clipping-012.htm
"You should see a 100x100 green square below. If you see any red, the
test has failed. This test is checking to make sure clip is applying to
all descendants."
My neighbour would ask "What's a 100x100, clip and descendants?"
The testcase has validation markup errors too.

--------------------

13- Expected results "green box on top of a blue box" need to be updated

http://test.csswg.org/suites/css2.1/20100917/html4/background-087.htm
http://test.csswg.org/suites/css2.1/20100917/html4/background-182.htm

--------------------

14- (Boris Zbarsky) A few table-anonymous-objects-0xx testcases use and
rely on more than 1 element sharing the same id attribute value. This is
not only invalid HTML but it can make strictly compliant HTML4 or DOM
compliant browsers refuse to do more than 1 element with the same id.

E.g.:

http://test.csswg.org/suites/css2.1/20100917/html4/table-anonymous-objects-021.htm
and
http://test.csswg.org/suites/css2.1/20100917/html4/table-anonymous-objects-022.htm

line 58  <div id="data" class="table">
line 70  <div id="data" style="display: table-row-group">

and the script relies on those id="data" to iterate the elements.

http://test.csswg.org/suites/css2.1/20100917/html4/table-anonymous-objects-023.htm
and
http://test.csswg.org/suites/css2.1/20100917/html4/table-anonymous-objects-024.htm

2 id="t"

line 27:    <table border="5"><tbody><tr><td id="t">Some
text</td></tr></tbody></table>

line 31:    <table border="5"><tbody><tr><td id="t">Some
text</td></tr></tbody></table>

Several table-anonymous-objects-0xx have also other smaller problems:
- validation markup errors (style without type attribute,
language="javascript", style block can not have an id, etc)
- Inline style is formally discouraged:
"Do not use the style attribute (inline styles) unless specifically
testing that attribute"
http://wiki.csswg.org/test/css2.1/format#body-content
- Because of onload="doTest()", you need to add
<meta http-equiv="Content-Script-Type" content="type">

--------------------

15- HTMLonly
http://test.csswg.org/suites/css2.1/20100917/html4/content-072.htm

The testcase should have HTMLonly flag.


I have a list of well over 100 testcases with less serious issues which
would nevertheless require some (text|code|layout) tuning.

I will be taking a 1 week break starting on monday 27th.

best regards from Montréal, Gérard
-- 
Contributions to the CSS 2.1 test suite:
http://www.gtalbot.org/BrowserBugsSection/css21testsuite/

CSS 2.1 test suite (RC1; September 17th 2010):
http://test.csswg.org/suites/css2.1/20100917/html4/toc.html

CSS 2.1 test suite contributors:
http://test.csswg.org/source/contributors/

Received on Friday, 24 September 2010 21:00:34 UTC