Re: CSS 2.1 test suite feedback: general test issues

> I've been through about 60% of the HTML4 tests in the 20100917 suite,
and have some feedback.
> This message is about general issues with the tests themselves.
> Many tests use the terms "text" and "line" interchangeably. In other,
"text" specifically refers to the text foreground (e.g. "this test
should be green"), and "line" to the background ("this line should be
green"). There should be consistency here. The tests that I've found
so
> far that have this ambiguity are:
> 	c15-ids-001 et al.
> 	at-charset-071
> 	first-line-pseudo-017 et al.
> 	matching-brackets-003
> 	margin-collapse-139 et al.

Hello Simon,

I agree with you that pass conditions should be more accurate and
overall consistent, coherent.

CSS2.1 Test Case Authoring Guidelines
4.1.1. The green paragraph
http://www.w3.org/Style/CSS/Test/guidelines.html#the-green

In some languages, "this line should be green" can refer to the text as
line is presumed or understood as line of text.

But there is no ambiguity when describing
"This line should have a green background"


> Similarly, some tests use the term "box", some "square" and some
"block". I'm assuming that a "box" and "block" may be rectangular, but
that a "square" must have sides of the same length. This is generally
clear.

I really think that "box" and "block" should be removed everywhere used.
Rectangle and square (geometrical shapes) should be used instead.

1- Better shape descriptions should be used when relevant: stripe, bar,
line.

2- Orientation descriptions should be used when relevant: horizontal
bar, vertical line, left side, upper-left corner, etc.

3- Relative dimension descriptions should be used when relevant: large,
tall, larger, taller, wide, narrow, small, etc.

4- Often, "Filler text" should be instead/rather "Text sample";
sometimes, it could be "Test target" or something like that.

> There's also ambiguity about whether these terms refer to a solid,
filled box, or a hollow, bordered box. I've found these tests with
ambiguity in this area:
> 	numbers-units-002
> 	border-width-applies-to-001
> 	positive-integer-001
> 	cascade-008
> 	border-left-width-095
> 	border-applies-to*

Again, I agree with you. It makes the test suite not entirely coherent,
not entirely consistent. It can easily create some hesitation in the
mind of testers.

"Filled box or just black-bordered box?"
(...)
"More relevant shape description"
http://lists.w3.org/Archives/Public/public-css-testsuite/2009Sep/0014.html


> The easiest tests to judge are those that use the "This test should be
green" technique. The hardest are those that use the descriptive form,
like "there should a blue box on the left, and an orange box on the
right, and the text must flow above the blue box and under the orange
box". We should take very opportunity to convert tests to the former
style.

Some testcases can be made simplier and more straightforward from a
tester point of view by converting them into "There should be a green
square". But for some others, it can be difficult, too difficult or not
suitable to convert.

> Ref tests are by far the easiest to judge (as long as I don't have to
scroll to see the entire test).
> Some tests use a different green color that others. It would be easier
if they all used the "green" color.

Writing black into a green filled/painted square can make such text
difficult to read: there is a color brightness/contrast standard on
this. This is a problem in some testcases: there is black color text
written inside a green area when it should have been white color text
instead.

I prefer white text color into a green area and black text color into a
lime area.

regards, Gérard
-- 
Contributions to the CSS 2.1 test suite:
http://www.gtalbot.org/BrowserBugsSection/css21testsuite/

CSS 2.1 test suite (RC2; October 1st 2010):
http://test.csswg.org/suites/css2.1/20101001/html4/toc.html

CSS 2.1 test suite contributors:
http://test.csswg.org/source/contributors/

Received on Sunday, 3 October 2010 19:11:03 UTC