W3C home > Mailing lists > Public > public-css-testsuite@w3.org > January 2010

t1508-c527-font-00-b.htm to t1508-c527-font-10-c.htm: Testcases pass condition are not self-evident

From: Gérard Talbot <css21testsuite@gtalbot.org>
Date: Mon, 25 Jan 2010 16:32:30 -0800
Message-ID: <0674e05884e0e6c63c3e3135939cd333.squirrel@cp3.shieldhost.com>
To: "fantasai" <fantasai.lists@inkedblade.net>
Cc: "Arron Eicholz" <arron.eicholz@microsoft.com>, "public-css-testsuite@w3.org" <public-css-testsuite@w3.org>
Hello all,

A few days ago, I took the CSS 2.1 test suite (511 testcases) with the
CSS 2.1 Conformance Test Harness with latest version of Konqueror.

When I arrived at those tests:

http://www.w3.org/Style/CSS/Test/CSS2.1/current/html4/t1508-c527-font-00-b.htm
http://www.w3.org/Style/CSS/Test/CSS2.1/current/html4/t1508-c527-font-01-b.htm
http://www.w3.org/Style/CSS/Test/CSS2.1/current/html4/t1508-c527-font-02-b.htm
http://www.w3.org/Style/CSS/Test/CSS2.1/current/html4/t1508-c527-font-03-b.htm
http://www.w3.org/Style/CSS/Test/CSS2.1/current/html4/t1508-c527-font-04-b.htm
http://www.w3.org/Style/CSS/Test/CSS2.1/current/html4/t1508-c527-font-05-b.htm
http://www.w3.org/Style/CSS/Test/CSS2.1/current/html4/t1508-c527-font-06-b.htm
http://www.w3.org/Style/CSS/Test/CSS2.1/current/html4/t1508-c527-font-07-b.htm
http://www.w3.org/Style/CSS/Test/CSS2.1/current/html4/t1508-c527-font-08-b.htm
http://www.w3.org/Style/CSS/Test/CSS2.1/current/html4/t1508-c527-font-09-b.htm
http://www.w3.org/Style/CSS/Test/CSS2.1/current/html4/t1508-c527-font-10-c.htm

I sincerely could not say that those tests were passed or failed simply
because those tests assumed that the tester has eyes which can see (and
know where they are to start with) respective baselines, measure a
difference of 9px and distinguish accordingly a difference of 12px,
18px, etc. "John Doe" and "Joe Schmuck" would never be able to say with
certainty that this gap here is 9px while this other gap over there is
definitely not 9px but must be 12px.

e.g.: "There should be 27px between baselines (9px gap between lines)."
and the baselines are not drawn, not visible, not identified...

Those tests have no measurement rules, no reference for comparison, no
substitute/compensation of some kind for px measurements (like a
screenshot or a reference for expected results).

Not realistic. Not reasonable.

Add to this the fact that some browsers may not even define font-size in
pixels in the settings/preferences but in pt.

regards, Gérard
-- 
Contributions to the CSS 2.1 test suite:
http://www.gtalbot.org/BrowserBugsSection/css21testsuite/

CSS 2.1 test suite (pre-alpha):
http://www.w3.org/Style/CSS/Test/CSS2.1/current/html4/

CSS 2.1 test suite contributors:
http://test.csswg.org/source/contributors/
Received on Tuesday, 26 January 2010 00:33:06 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Monday, 20 September 2010 17:52:01 GMT