- From: Gérard Talbot <css21testsuite@gtalbot.org>
- Date: Tue, 6 Dec 2011 13:59:54 -0800
- To: "Public CSS test suite mailing list" <public-css-testsuite@w3.org>
Hello all,
I am open to feedback, criticisms and any comments you may have on this
email.
(Disclaimer: I personally have to fix a few of my own testcases so that
I can comply with my very own recommendations here! Duh!)
1-
Adequate font-size for :first-letter testcases
**********************************************
:first-letter testcases should all be using a much bigger font-size:
say, at least 32px
Minimum 2em; recommended: 3em;
eg.
http://test.csswg.org/suites/css2.1/20110323/html4/first-letter-001.htm
http://test.csswg.org/suites/css2.1/20110323/html4/first-letter-dynamic-001.htm
http://test.csswg.org/suites/css2.1/20110323/html4/first-letter-dynamic-003a.htm
http://test.csswg.org/suites/css2.1/20110323/html4/first-letter-dynamic-003b.htm
http://test.csswg.org/suites/css2.1/20110323/html4/first-letter-nested-001.htm
2-
font units and line-height
**************************
3pt == 4px; so best is to use a factor of 3 for pt unit: ideal is
12pt (or 15pt or 18pt, etc) for parent font size.
5ex (ahem) = 4em; so best is to use a factor of 5 for ex unit;
ideal is 20px (or 30px or 40px, etc) for parent font size.
The ideal is to use a value which is going to be convertible to pixel
unit into whole integer, without creating fraction.
font: 1.25em/1.4 serif;
will initially be resorting to a font-size of 20px and a line-height of
28px.
For optimal convertability, even values are preferred because creating
more predictable layout.
3-
Adequate margin-left for list-marker testcases
**********************************************
Horizontal space should be made available in testcases to view bullet
list-marker, in particular in 'list-style-position: outside' testcases.
This should be part of pass condition of all list-style testcases.
Minimum 1em; recommended: 2em.
Examples:
http://test.csswg.org/suites/css2.1/20110323/html4/after-content-display-003.htm
http://test.csswg.org/suites/css2.1/20110323/html4/universal-selector-005.htm
4-
outline borders fully viewable
******************************
Parts of the outline can be clipped on the left and on the right.
The full width of outline should be viewable, visible in testcases,
furthermore if tests require to examine them or even compare them with a
reference border.
Recommendation: when outline-width is, say, 20px, then set margins to
horizontal margins to 20px too.
E.g.:
http://test.csswg.org/suites/css2.1/20110323/html4/outline-001.htm
<style type="text/css">
#div1
{
outline: green;
outline-style: dotted;
outline-width: 15px;
}
</style>
</head>
<body>
<p>Test passes if there is a dotted outline that goes around the
following 'Sample Text' and the "Filler Text" below is
overlapped by the outline.</p>
<div id="div1">Sample Text</div>
<div>Filler Text</div>
would become
<style type="text/css">
div {margin: auto 15px;}
#div1
{
outline: green;
outline-style: dotted;
outline-width: 15px;
}
</style>
5-
"Filler text" versus "Text sample"
**********************************
"Filler text" should be for background.
"Text sample" or "Text example" should be for testing.
(bad)
<div>Filler text</div>
<div>Filler text</div>
<div>Filler text</div>
(good)
<div>1st text sample</div>
<div>2nd text sample</div>
<div>3rd text sample</div>
(good)
<div>1st text example</div>
<div>2nd text example</div>
<div>3rd text example</div>
6-
Distinguishible text content when using Ahem font
*************************************************
When editing a text node which is going to be using Ahem font, then
give preference to
1234567890
instead of/rather than
XXXXXXXXXX
, especially if counting the characters is important in the testcase;
that way, test reviewer, test author when revisiting testcase much
later, web authors do not have to count anything. The string
"1234567890" can also be much easier to find or to test with web
authoring debugging tools like IE Developer Tools, Firefox's DOM
inspector, Chrome Web Inspector, Safari Web Inspector, Opera DragonFly,
Konqueror's DOM treeviewer.
Another useful manner to edit text nodes is to give text content an
useful, meaningful, descriptive value in relationship with the purpose
of the test: eg.
<span>MustBeAbove</span>
is going to be much more helpful, useful and appreciated than
<span>XXXXXXXXXXX</span>. Because it contributes to self-explanatory sense
of testcases while editing distinguishable text content.
Use distinguishible, unique text content whenever possible and wherever
suitable so that examining the source code can be easier, so that the
efforts of understanding the concepts - sometimes difficult ones - and
the testcase itself (and its complexity) can be less demanding
E.g.:
(undifferentiated therefore less recommendable)
<p>x <span>xxxxx xxxxx</span> x</p>
versus
(differentiated therefore more recommendable)
<p>a <span>ceikl mnors</span> u</p>
E.g.:
<div>shold be overlapped<br>
<span>bleed over 1st line</span></div>
http://test.csswg.org/suites/css2.1/nightly-unstable/html4/border-padding-bleed-001.htm
If a green rectangle indicating a passed test must have a text content,
then using "PASS" is more logical and useful than "X". If a green
rectangle indicating a passed test must have a text content, then using
"FAIL" (for the overlapped red element) is more logical and useful than
"X".
e.g:
http://test.csswg.org/suites/css2.1/nightly-unstable/html4/absolute-non-replaced-width-026.htm
7-
Correct usage of 'border-collapse: collapse'
********************************************
It is okay to declare 'border-collapse: collapse' when table and/or
sub-table components have borders which require, by the testcase, to be
collapsed. It is *not* okay to use 'border-collapse: collapse' in order
to neutralize or to set to zero the border-spacing value.
There are many testcases where 'border-collapse: collapse' is improperly
used.
8-
Avoid single cell with single row tables
****************************************
Mono-cell in mono-table-row table in table layout rendering testcases is
to be avoided because those are not very helpful and not very reliable.
It may lead to false positives. And they are not realistical usage of
tables either.
eg
http://test.csswg.org/suites/css2.1/20110323/html4/background-position-applies-to-004.htm
is passed by IE9 and Chrome 14 but
http://test.csswg.org/suites/css2.1/20110323/html4/table-backgrounds-bs-row-001.htm
and
http://test.csswg.org/suites/css2.1/20110323/html4/table-backgrounds-bs-row-002.htm
are failed by IE9 and Chrome 14.
Additional testcase: http://the-dees.webs.com/iepp1/55-tr-bg-position.html
8b) best or better, more recommendable testing of inline-block when its
content does not have any block containers.
e.g.
(bad or less good)
<div>
<span style="display: inline-block;">text sample</span>
</div>
(good or better)
<div>
<span style="display: inline-block;">
<span style="display: block;">text sample1</span>
<span style="display: block;">text sample2</span>
</span>
</div>
9-
Recommended shape descriptors
*****************************
The whole CSS 2.1 test suite could be arranged to only use 6 shape
descriptors: rectangle, square, stripe, bar, line, grid.
Sometimes, mentioning "a blue square" is sufficient; sometimes, it will be
necessary for the tester to know if it's a filled blue square or a white
square with a blue border.
Shape descriptors should be furthermore specified depending on the needs
of a testcase:
"line" vs "green line" vs "vertical green line" vs "short vertical green
line"
eg:
http://test.csswg.org/suites/css2.1/20101027/html4/border-right-applies-to-002.htm
a box, a black box, a black square, a filled black square, a small
filled black square, etc.
http://test.csswg.org/suites/css2.1/20101027/html4/border-right-color-013.htm
Example
http://test.csswg.org/suites/css2.1/20110323/html4/abspos-001.htm
says "This page should have a single green rectangle (...)"
when it would be preferable to say
"There should be a short green stripe (...)"
eg How big/large/tall should be a "box"?
http://test.csswg.org/suites/css2.1/20110323/html4/absolute-replaced-height-008.htm
http://test.csswg.org/suites/css2.1/20110323/html4/absolute-replaced-height-009.htm
http://test.csswg.org/suites/css2.1/20110323/html4/absolute-replaced-height-016.htm
http://test.csswg.org/suites/css2.1/20110323/html4/absolute-replaced-height-022.htm
http://test.csswg.org/suites/css2.1/20110323/html4/absolute-replaced-height-023.htm
http://test.csswg.org/suites/css2.1/20110323/html4/absolute-replaced-height-029.htm
http://test.csswg.org/suites/css2.1/20110323/html4/absolute-replaced-height-030.htm
I think we should totally ban the word "box" and "block" in the
pass/fail condition sentence of all testcases. For most people, a "box"
is 3-dimensional object to begin with.
Reuse terms, expressions used in the general public, in common places is
generally good.
Example: "bathroom tiles" is an excellent expression for use in the test
suite.
10-
Sentences, statements descriptions
**********************************
(good) This sentence should have green text.
(not best, ambiguous) This sentence should be green.
(good) This text should be green.
(bad) This line should be green.
(good) This sentence should have a green background.
Ambiguous:
http://test.csswg.org/suites/css2.1/20110323/html4/background-image-005.htm
11-
Removable and emboldenable words
********************************
Removable
=========
visible
viewable
appearing
displayed
you can see
in this page
on this page
below
below this line (eg
http://test.csswg.org/suites/css2.1/20110323/html4/replaced-intrinsic-002.htm)
below this sentence
below this paragraph (eg
http://test.csswg.org/suites/css2.1/20110323/html4/abspos-001.htm)
under this paragraph (eg
http://test.csswg.org/suites/css2.1/20110323/html4/tables-002.htm)
after this
It is presumed that testers taking the test suite are not blind and do
not have an incapacitating daltonism condition or some kind of color
vision illness or incapacity. Mentioning a color implies by itself that
it can be viewed, it is visible by senses.
eg (bad) "Test passes if there is a green box visible and there is no red
visible on the page."
http://test.csswg.org/suites/css2.1/20101210/html4/clip-101.htm
Why remove such words? Because they are useless, empty-meaning,
pleonastic or self-evident or irrelevant words; they can be safely
removed without affecting understanding of pass/fail conditions.
Emboldenable
============
no red
green square (with relevant shape descriptors: {rectangle, square,
stripe, bar, line, grid}) [color][shape descriptor]
identical
the same
look the same
same order
in the center
horizontally centered
not or NOT (eg not slanted)
eg not adjoining
http://test.csswg.org/suites/css2.1/20110111/html4/border-spacing-applies-to-007.htm
filled square
hollow square
[color] square with [color] borders
identical colored bands
upper|lower left|right corner
top|bottom left|right corner
is in upper-left corner
fills the upper-left corner
inside the box/square
on the same line
same color
same dimensions
same size
same width
same height
same horizontal position
same vertical position
larger (horizontal axis)
taller (vertical axis)
protude out of
(top|bottom) of {rectangle, square, stripe, bar, line, grid} flush with
(top|bottom) of {rectangle, square, stripe, bar, line, grid} level with
(top|bottom) of {rectangle, square, stripe, bar, line, grid} aligned with
(no space|no gap) between
touching
Why emboldening such words?
The most important pass conditions are easy to read, to find; this
help eye/reading scannability, text skimmability, faster test checking.
Also removable: "exact" and "perfect" from "exact square", "perfect
square".
eg exact square:
http://test.csswg.org/suites/css2.1/20110323/html4/replaced-intrinsic-002.htm
12-
"Test passes if there is (are) ..."
***********************************
Most of the testcases now start with "Test passes if there is (are) ..."
which is probably the best introductory formulation for testcases;
reusing consistently such formulation makes the reading of testcases
more predictable.
13-
Avoid horizontal scrollbar generated with a window viewport of
1000px wide (screen resolution)
E.g.:
http://test.csswg.org/suites/css2.1/20110323/html4/vertical-align-121.htm
14-
Use description wisely
**********************
http://test.csswg.org/suites/css2.1/20110323/html4/cursor-014.htm
Test passes if a cursor that indicates westward movement appears when ...
(better) Test passes if a double-headed horizontal arrow appears when ...
15-
Avoid technical terms, otherwise describe them
**********************************************
(bad) "You should see a 100x100 green square below."
http://test.csswg.org/suites/css2.1/20100917/html4/clipping-002.htm
(bad) "a 10em box"
(bad) "The two sweets"
http://test.csswg.org/suites/css2.1/20110323/html4/c5510-ipadn-000.htm
(bad) "multi-bit alpha channel"
http://test.csswg.org/suites/css2.1/20110323/html4/background-alpha-001.htm
16-
Graphical layout of objects to compare
**************************************
If measurements are performed by comparing with a referring
{stripe, bar, square, rectangle, line}, then best is to align
horizontallly the tested geometrical shapes if their height is the
object of the test and to align vertically the tested geometrical shapes
if their width is the object of the test.
The general idea is to arrange graphical layout of objects in a way that
eases the task of the tester.
17-
Reuse CSS 2.1 terminology
*************************
In the text assert and in title, reuse the spec terminology, vocabulary
and do not add new, unknown terminology.
(bad) Shrink-to-fit height
http://test.csswg.org/suites/css2.1/20110111/html4/absolute-non-replaced-height-002.htm
(good) Height is based on content.
"shrink-wraps", "shrink-wrapping" are not defined anywhere in the spec.
Ancestor is preferable/better to ascendant.
(bad or even incorrect) "induce a style"
http://test.csswg.org/suites/css2.1/20110323/html4/table-column-rendering-004.htm
18-
Use meaningful, helpful, semantical id or class attributes
**********************************************************
Bad, otherwise not ideal, not recommendable:
(bad) class="class1"
(bad) class="class2"
(bad) class="one"
(bad) class="two"
(bad) class="a"
(bad) class="b"
(bad) id="div1"
(bad) id="div2"
(bad) id="span1"
(bad) id="span2"
(bad) id="one"
(bad) id="two"
(bad) id="a"
(bad) id="b"
id="test" along with function test()
(good) class="abs-pos-parent"
(good) id="rel-pos-grand-parent"
(good) id="containing-block"
(good) class="blue-float-left"
(good) id="red-overlapped"
(good) id="green-overlapping"
(good) id="reference"
(good) id="control"
(good) id="clearing"
(good) id="following-sibling"
(good) class="adjacent-sibling"
Good identifiers are those which are
- descriptive with regards to the working logic of the testcase,
- descriptive of the design logic of the testcase or
- descriptive of the respective position in the containment hierarchy or
- descriptive of the respective position in the positioning hierarchy.
Identifiers should describe the role of an element inside the building
logic of a testcase.
19-
Include calculations and/or explanations into comments of the testcase
***********************************************************************
Eg.
http://test.csswg.org/suites/css2.1/20110323/html4/position-absolute-percentage-inherit-001.htm
http://test.csswg.org/suites/css2.1/20110323/html4/margin-collapse-clear-013.htm
20- Try to make the most important pass/fail condition of a testcase the
most important visual aspect of the page and the only/sole visual aspect
of the page
21-
Sufficient background/foreground color contrast/brightness
**********************************************************
Some tests inadvertedly, awkwardly use 2 dark colors. Or they write with a
dark color inside another dark color area.
eg (bad)
http://test.csswg.org/suites/css2.1/20110323/html4/column-border-002.htm
eg (bad)
http://test.csswg.org/suites/css2.1/20110323/html4/first-line-inherit-002.htm
22-
Graphically arrange layout to help tester
*****************************************
The testcase should be built in a way that, when it fails, it is
graphically clear and obvious that it fails:
In
http://test.csswg.org/suites/css2.1/20110323/html4/block-non-replaced-width-008.htm
http://test.csswg.org/suites/css2.1/nightly-unstable/html4/column-visibility-004.htm
the amount of red is equal to the amount of green. So, in case of a test
failure, the tester will easily notice the failure.
(bad) http://test.csswg.org/suites/css2.1/20110323/html4/max-height-017.htm
An UA which does not implement max-height is going to show 2 squares
whose differences of height is going to be 1px only: for many humans, a
hardly noticeable difference. A machine running software would mark this
as fail though.
23-
Spelling
********
descendent (eg descendent-selector-00? testcases), PREREQUSITE,
strips (table-borders-002 and 004)
24-
Containing block is not a CSS box
*********************************
The concept of containing block is not a CSS box; it's nothing more than
a rectangle formed by ( see section 10.1 as there are more than 1
answer) which has a width, a height, a top edge, a left edge, a bottom
edge and a right edge. It serves only to position positioned elements.
So, padding of containing block and content edge of containing block are
out of spec terms and should not be used.
25-
Title versus text assert coherence
**********************************
The title of a testcase should never contradict or oppose the assert
text. Coherence, consistence, clarity at all times.
There are *_hundreds_* of testcases where this happens. The title says -
example given
"Fixed position elements impact on later siblings"
but the assert says
"Fixed positioned elements do not impact later siblings."
http://test.csswg.org/suites/css2.1/20110323/html4/position-absolute-001.htm
The thing is that even in the listing of testcases, users will more
likely see the title rather than the assert because of smaller font-size
used for the assert. In any case, a blatant contradiction like this is
never going to help users to understand CSS 2.1. Some people think and
say that CSS 2.1 is tough to read, hard to understand, without enough
examples, etc... and now they could add that the CSS 2.1 test suite does
not help as much as it could... by being consistently clear at all
times.
The title of a testcase should probably only lists the most important
properties or concept involved.
<title>CSS Test: border-collapse: collapse - adjacent cells with same
border-style and border-width</title>
or
<title>CSS Test: border conflict element - adjacent cells with same
border-style and border-width</title>
26-
Reusing the same rendered layout
********************************
When creating a serie of testcases, try to use and reuse the same
rendered layout. It not only helps the tester but it can reduce the number
of reftests needed to create later.
Bad:
Stack of 5 colored bands
------------------------
margin-collapse-106, 112: a stack of 5 bands which are from top to
bottom: orange, yellow, orange, lime, orange
margin-collapse-110, 111: a stack of 5 bands which are from top to
bottom: orange, lime, orange, yellow, orange
margin-collapse-114: a stack of 5 bands which are from top to bottom:
yellow, orange, aqua, lime, orange
Good:
Stack of blue, orange, yellow squares
-------------------------------------
http://test.csswg.org/suites/css2.1/nightly-unstable/html4/z-index-stack-001.htm
http://test.csswg.org/suites/css2.1/nightly-unstable/html4/z-index-stack-002.htm
http://test.csswg.org/suites/css2.1/nightly-unstable/html4/z-index-stack-003.htm
(albeit dimensions need to be uniformized to 100px by 100px and boxes
should be replaced with squares)
27- Try to make testcase use a reasonable font-size: minimum 16px,
recommended 20px with a 4-8px leading. eg font: 1.25em/1.4 serif;
Try to make testcase use a font-size and a line height with units which
makes it text-size-scalable. em and percentages for font-size is good
and number is good for line-height
eg.
less recommendable:
http://test.csswg.org/suites/css2.1/nightly-unstable/html4/border-padding-bleed-001.htm
less recommendable:
http://test.csswg.org/suites/css2.1/nightly-unstable/html4/position-absolute-percentage-inherit-001.htm
better, more recommendable:
http://test.csswg.org/suites/css2.1/nightly-unstable/html4/vertical-align-117a.htm
better, more recommendable:
http://test.csswg.org/suites/css2.1/nightly-unstable/html4/separated-border-model-007.htm
28-
Consistency of vocabulary, terminology, expressions used, shape descriptors
*****************************************************************
box is [color name]
http://test.csswg.org/suites/css2.1/20110323/html4/background-image-cover-001.htm
background of [text] is [color name]
http://test.csswg.org/suites/css2.1/20110323/html4/background-transparency-001.htm
At all times, you want, aim for a test where pass/fail condition
sentence is unequivocal, disambiguated, tries to establish a 1 to 1
relation.
regards, Gérard
--
Contributions to the CSS 2.1 test suite:
http://www.gtalbot.org/BrowserBugsSection/css21testsuite/
CSS 2.1 Test suite RC6, March 23rd 2011:
http://test.csswg.org/suites/css2.1/20110323/html4/toc.html
CSS 2.1 test suite harness:
http://test.csswg.org/harness/
Contributing to to CSS 2.1 test suite:
http://www.gtalbot.org/BrowserBugsSection/css21testsuite/web-authors-contributions-css21-testsuite.html
Received on Tuesday, 6 December 2011 22:00:33 UTC