- From: Gérard Talbot <css21testsuite@gtalbot.org>
- Date: Mon, 11 Jun 2018 18:26:55 -0400
- To: Public CSS Test suite mailing list <public-css-testsuite@w3.org>
Hello all,
Since 2010, I have examined, checked and reviewed hundreds of CSS tests.
I wrote down all kinds of notes about what I found. The following
guidelines are my recommendations. They are listed in no particular
order. They are not some kind of biblical 39 commandements or anything
that everyone must follow. Heck, some items could be debattable or minor
or a matter of taste or plain wrong. And some of my own tests might even
be in conflict with my own guidelines!
This is a follow-up/continuation of
28 proposals to improve testcase writing guidelines
http://lists.w3.org/Archives/Public/public-css-testsuite/2011Dec/0004.html
Overall, I am open to feedback, criticisms and any comments you may have
on this document.
- - - - - - - - -
1-
Adequate font-size for :first-letter testcases
**********************************************
:first-letter testcases should all be using a much bigger font-size:
say, at least 32px
Minimum 2em; recommended: 3em;
eg.
http://test.csswg.org/suites/css2.1/20110323/html4/first-letter-001.htm
http://test.csswg.org/suites/css2.1/20110323/html4/first-letter-dynamic-001.htm
http://test.csswg.org/suites/css2.1/20110323/html4/first-letter-dynamic-003a.htm
http://test.csswg.org/suites/css2.1/20110323/html4/first-letter-dynamic-003b.htm
http://test.csswg.org/suites/css2.1/20110323/html4/first-letter-nested-001.htm
http://test.csswg.org/suites/css2.1/nightly-unstable/html4/first-letter-quote-001.htm
http://test.csswg.org/suites/css2.1/nightly-unstable/html4/first-letter-quote-005.htm
http://test.csswg.org/suites/css2.1/nightly-unstable/html4/first-letter-selector-018.htm
Better
http://test.csswg.org/suites/css2.1/nightly-unstable/html4/first-letter-001.htm
http://test.csswg.org/suites/css2.1/nightly-unstable/html4/first-letter-nested-001.htm
2-
font units and line-height
**************************
3pt == 4px; so best is to use a factor of 3 for pt unit: ideal is
12pt (or 15pt or 18pt, etc) for parent font size.
5ex (ahem) = 4em; so best is to use a factor of 5 for ex unit;
ideal is 20px (or 30px or 40px, etc) for parent font size.
The minimal font-size for all testcases involving ex unit and Ahem font
should be 20px; and then a factor of 10: 20px, 30px, 40px, etc.
The ideal is to use a value which is going to be convertible to pixel
unit into whole integer, without creating fraction.
font: 20px/1.5 serif;
will initially be resorting to a font-size of 20px and a line-height of
30px
For optimal convertability, even values are preferred because creating
more
predictable layout.
Leading outside of content area should be dividable by 2 without a
remainer.
font: 25px/50px Ahem;
is going to create problems as 25px is not dividable by 2 without a
remainer.
Good is:
font: 25px/51px Ahem;
because 26px is dividable by 2 without a remainer.
3-
Adequate margin-left for list-marker testcases
**********************************************
Horizontal space should be made available in testcases to view bullet
list-marker, in particular in 'list-style-position: outside' testcases.
This should be part of pass condition of all 'list-style' testcases.
Minimum 1em; recommended: 2em.
Examples:
http://test.csswg.org/suites/css2.1/20110323/html4/after-content-display-003.htm
http://test.csswg.org/suites/css2.1/20110323/html4/universal-selector-005.htm
4-
Outline borders fully viewable
******************************
Parts of the outline can be clipped on the left and on the right.
The full width of outline should be viewable, visible in testcases,
furthermore
if tests require to examine them or even compare them with a reference
border.
Recommendation: when outline-width is, say, 20px, then set all 4 margins
to 20px too.
E.g.:
http://test.csswg.org/suites/css2.1/20110323/html4/outline-001.htm
<style type="text/css">
#div1
{
outline: green;
outline-style: dotted;
outline-width: 15px;
}
</style>
</head>
<body>
<p>Test passes if there is a dotted outline that goes around the
following 'Sample Text' and the "Filler Text" below is overlapped by the
outline.</p>
<div id="div1">Sample Text</div>
<div>Filler Text</div>
would become
<style type="text/css">
div {margin: 15px;}
#div1
{
outline: green;
outline-style: dotted;
outline-width: 15px;
}
</style>
5-
"Filler text" versus "Text sample"
**********************************
"Filler text" should be for background.
(good)
http://test.csswg.org/suites/css2.1/20110323/html4/background-bg-pos-207.htm
"Text sample" or "Text example" should be for testing.
(good)
http://test.csswg.org/suites/css2.1/20110323/html4/content-type-000.htm
(bad)
<div>Filler text</div>
<div>Filler text</div>
<div>Filler text</div>
(good)
<div>1st text sample</div>
<div>2nd text sample</div>
<div>3rd text sample</div>
(good)
<div>Text example 1</div>
<div>Text example 2</div>
<div>Text example 3</div>
6-
Differentiable text content when using Ahem font
************************************************
When editing a text node which is going to be using Ahem font, then
give preference to
1234567890
instead of/rather than
XXXXXXXXXX
, especially if counting the characters is important in the testcase;
that way,
test reviewer, test author when revisiting testcase much later, web
authors do
not have to count anything. The string "1234567890" can also be much
easier to
find or to test with web authoring debugging tools like IE Developer
Tools,
Firefox's DOM inspector, Chrome Web Inspector, Safari Web Inspector,
Opera
DragonFly, Konqueror's DOM treeviewer.
Another useful manner to edit text nodes is to give text content an
useful,
meaningful, descriptive value in relationship with the purpose of the
test: eg.
<span>MustBeAbove</span>
is going to be much more helpful, useful and appreciated than
<span>XXXXXXXXXXX</span>. Because it contributes to self-explanatory
sense
of testcases while editing differentiable text content.
Use differentiable, unique text content whenever possible and wherever
suitable so that examining the source code can be easier, so that the
efforts of
understanding the concepts - sometimes difficult ones - and the testcase
itself (and its complexity) can be less demanding
E.g.:
(undifferentiated therefore less recommendable) <p>x <span>xxxxx
xxxxx</span> x</p>
versus
(differentiated therefore more recommendable) <p>a <span>ceikl
mnors</span> u</p>
E.g.:
<div>shuldboverlaPPed<br>
<span>bleedover1stline</span></div>
http://test.csswg.org/suites/css2.1/nightly-unstable/html4/border-padding-bleed-001.htm
If a green rectangle indicating a passed test must have a text content,
then
using "PASS" is more logical and useful than "X". If a green rectangle
must overlap a red rectangle with a text content, then using "FAIL" is
more
logical and useful than "X". Again, this contributes to make source text
content self-explanatory.
7-
Correct usage of 'border-collapse: collapse'
********************************************
It is okay to declare 'border-collapse: collapse' when table and/or
sub-table components have borders which require, by the testcase, to be
collapsed. It is *not* okay to use 'border-collapse: collapse' in order
to neutralize or to set to zero the border-spacing value. There are
currently many testcases where 'border-collapse: collapse' is improperly
used.
Also, if border-collapse: collapse must be used, then best is to use
even border-width values so that fractional pixel situations can be
avoided.
"
User agents must find a consistent rule for rounding off in the case of
an odd
number of discrete units (screen pixels, printer dots).
"
17.6.2 The collapsing border model
http://www.w3.org/TR/CSS21/tables.html#collapsing-borders
8-
Avoid single cell with single row tables
****************************************
Mono-cell in mono-table-row table in table layout rendering testcases is
to be
avoided because those are not very helpful and not very reliable. It may
lead to false positives. Such are not realistical usages of tables and
are not typical webpage scenarios either.
eg
http://test.csswg.org/suites/css2.1/20110323/html4/background-position-applies-to-004.htm
is passed by IE9 and Chrome 14 but
http://test.csswg.org/suites/css2.1/20110323/html4/table-backgrounds-bs-row-001.htm
and
http://test.csswg.org/suites/css2.1/20110323/html4/table-backgrounds-bs-row-002.htm
are failed by IE9 and Chrome 14.
Additional testcase:
http://the-dees.webs.com/iepp1/55-tr-bg-position.html
9-
How to test inline-block
************************
best or better, more recommendable testing of inline-block when its
content does not have any block containers.
e.g.
(bad or less good)
<div>
<span style="display: inline-block;">text sample</span>
</div>
(good or better)
<div>
<div style="display: inline-block;">
<span style="display: block;">text sample1</span>
<span style="display: block;">text sample2</span>
</div>
</div>
Minimal code to use with 'display' set to inline-block
http://www.gtalbot.org/BrowserBugsSection/css21testsuite/inline-block-minimal-code-2.html
Baseline-alignment of inline-blocks:
http://test.csswg.org/suites/css2.1/nightly-unstable/html4/vertical-align-baseline-004a.htm
http://test.csswg.org/suites/css2.1/nightly-unstable/html4/vertical-align-baseline-006a.htm
10-
Recommended shape descriptors
*****************************
Rectangle, square, stripe, bar, line, grid.
A stripe should be a short bar. A bar should be a long 1em line. A ideal
rectangle should be 200px wide by 100px tall or 300px wide by 100px
tall.
An ideal square is 100px by 100px.
The test suite often uses unsuited, unclear terms: "line", "box",
"block": in the end, there is no reliability, no common sense.
http://test.csswg.org/suites/css2.1/20110323/html4/abspos-001.htm
refers to a "rectangle" when I would use "stripe"
http://test.csswg.org/suites/css2.1/20110323/html4/abspos-024.htm
refers to a "green box" while
http://test.csswg.org/suites/css2.1/20110323/html4/abspos-028.htm
refers to a "green bar".
http://test.csswg.org/suites/css2.1/20110323/html4/background-image-cover-001.htm
refers to a blue box!... when it obviously should be a blue bar.
http://test.csswg.org/suites/css2.1/20110323/html4/c565-list-pos-000.htm
refers to a rectangle when I think it should mention a stripe.
http://test.csswg.org/suites/css2.1/20110323/html4/background-image-005.htm
"This line should be green"... but after a few hundreds of tests, we
have
no certainty on what "This line should be green" actually means. The
text
should be green? or the background of the line should be green?
Sometimes, mentioning "a blue square" is sufficient; sometimes, it will
be
necessary for the tester to know if it's a filled blue square or a white
square with a blue border.
Shape descriptors should be furthermore specified depending on the needs
of a test (from general to more specified):
line vs green line vs vertical green line vs short vertical green line
eg:
http://test.csswg.org/suites/css2.1/20110323/html4/border-right-applies-to-002.htm
(from general to more specified)
a square, a black square, a filled black square, a filled black
square on the right, etc.
http://test.csswg.org/suites/css2.1/20110323/html4/border-right-color-013.htm
Example
http://test.csswg.org/suites/css2.1/20110323/html4/abspos-001.htm
says "This page should have a single green rectangle below this
paragraph and no red."
when it would be preferable to say
"Test passes if there is a short green stripe and *no red*."
One danger of overspecifying shape descriptor unneedlessly, causing
pass/fail sentence to become long and arduous to read:
eg: "short vertical solid continuous line" may be over-doing,
overdescriptive.
eg How big/large/tall should be a "box"?
http://test.csswg.org/suites/css2.1/20110323/html4/absolute-replaced-height-008.htm
http://test.csswg.org/suites/css2.1/20110323/html4/absolute-replaced-height-009.htm
http://test.csswg.org/suites/css2.1/20110323/html4/absolute-replaced-height-016.htm
http://test.csswg.org/suites/css2.1/20110323/html4/absolute-replaced-height-022.htm
http://test.csswg.org/suites/css2.1/20110323/html4/absolute-replaced-height-023.htm
http://test.csswg.org/suites/css2.1/20110323/html4/absolute-replaced-height-029.htm
http://test.csswg.org/suites/css2.1/20110323/html4/absolute-replaced-height-030.htm
I think we should ban the word "box" in all testcases. For most people,
a box is 3-dimensional object to begin with.
Reusing terms, expressions used in the general public, in common places
is
generally good.
Example:
"bathroom tiles" is an excellent expression for use in the test suite.
eg.:
http://test.csswg.org/suites/css2.1/20110323/html4/min-width-036.htm
"Test passes if there is a box below."
but almost any rectangular shape would be a passed test.
11-
Sentences, statements descriptions
**********************************
(good) This sentence should have green text.
(not best, ambiguous) This sentence should be green.
(good) This text should be green.
(bad) This line should be green.
(bad)
http://test.csswg.org/suites/css2.1/20110323/html4/first-line-pseudo-017.htm
(bad)
http://test.csswg.org/suites/css2.1/20110323/html4/first-line-pseudo-018.htm
(good) This sentence should have a green background.
Ambiguous:
http://test.csswg.org/suites/css2.1/20110323/html4/background-image-005.htm
http://test.csswg.org/suites/css2.1/20110323/html4/c531-color-000.htm
http://test.csswg.org/suites/css2.1/20110323/html4/c532-bgcolor-000.htm
(bad)
http://test.csswg.org/suites/css2.1/nightly-unstable/html4/first-line-001.htm
12-
Removable and emboldenable words
********************************
Removable
=========
visible
viewable
appearing
present
anywhere
displayed
you can see
(
http://test.csswg.org/suites/compositing-1_dev/nightly-unstable/html/mix-blend-mode-blending-with-sibling.htm
)
You should see
(http://test.csswg.org/suites/css2.1/20110323/html4/inlines-013.htm)
you can view
in this page
on this page
on the screen
(
http://test.csswg.org/suites/compositing-1_dev/nightly-unstable/html/mix-blend-mode-blending-with-sibling.htm
)
below
below this line (eg
http://test.csswg.org/suites/css2.1/20110323/html4/replaced-intrinsic-002.htm)
after this line
below this sentence
after this sentence
below this paragraph (eg
http://test.csswg.org/suites/css2.1/20110323/html4/abspos-001.htm)
under this paragraph (eg
http://test.csswg.org/suites/css2.1/20110323/html4/tables-002.htm)
in the next paragraph
after this
following
which follows
It is presumed that testers taking the test suite are not blind and do
not
have an incapacitating daltonism condition or some kind of color vision
illness or incapacity. Mentioning a color implies by itself that it can
be viewed, it is visible by senses.
eg (bad) "Test passes if there is a green box visible and there is no
red
visible on the page."
http://test.csswg.org/suites/css2.1/20110323/html4/clip-101.htm
Since 95% (or so) of all tests starts with a sentence stating the
pass and fail condition, then "below", "below this line", "after this
sentence", "under this paragraph", etc.. is evident, pleonastic.
Why remove such words? Because they are useless, empty-meaning,
pleonastic or self-evident or irrelevant words; they can be safely
removed without affecting understanding of pass/fail conditions.
Anything that shortens the pass/fail conditions sentence of a test,
without compromising its understanding for a tester, helps. This is
particularly true if a test suite has 1000 tests or more!
Emboldenable
============
Emboldened words attrack reading, promotes text skimmability, helps text
scannability, faster test checking. Those words should identify clearly
and quickly the most important aspect of a pass/fail condition or the
determinant relationship between 2 objects to compare in the testcase or
the most determining pass/fail component of a test. All this is
especially true if a testsuite has over 1000 testcases. So, here's my
list of emboldenable words:
no red
green square (with relevant shape descriptors: {rectangle, square,
stripe, bar, line, grid}) [color][shape descriptor]
identical
the same
look the same
in the center
horizontally centered
not or NOT (eg not slanted)
eg not adjoining
http://test.csswg.org/suites/css2.1/20110111/html4/border-spacing-applies-to-007.htm
filled {rectangle, square, stripe, bar, line, grid}
hollow {rectangle, square, stripe, bar, line, grid}
[color] square with [color] borders
identical colored bands
is in [upper|lower]-[left|right] corner
fills the [upper|lower]-[left|right] corner
inside the square
on the same line
same color
same dimensions
same order
same size
same width
same height
same horizontal position
same vertical position
larger (horizontal axis)
taller (vertical axis)
protude out of
[ top | bottom ] of {rectangle, square, stripe, bar, line, grid} flush
with
[ top | bottom ] of {rectangle, square, stripe, bar, line, grid} level
with
[ top | bottom ] of {rectangle, square, stripe, bar, line, grid} aligned
with
[ no space | no gap ] between
touching
is flush with
is aligned with
Why emboldening such words?
The most important pass conditions are easy to read, to find; this
help eye/reading scannability, text skimmability, faster test checking.
Also removable: "exact" and "perfect" from "exact square", "perfect
square".
eg exact square:
http://test.csswg.org/suites/css2.1/20110323/html4/replaced-intrinsic-002.htm
"perfectly rectangular"
http://test.csswg.org/suites/css2.1/20110323/html4/blocks-011.htm
http://test.csswg.org/suites/css2.1/20110323/html4/blocks-012.htm
http://test.csswg.org/suites/css2.1/20110323/html4/blocks-014.htm
http://test.csswg.org/suites/css2.1/20110323/html4/blocks-015.htm
13-
"Test passes if there is (are) ..."
***********************************
Most of the testcases now start with "Test passes if there is (are) ..."
which
is probably the best introductory formulation for testcases; reusing
consistently such formulation makes the reading of testcases more
predictable.
14-
Horizontal scrollbar
********************
Avoid horizontal scrollbar generated with a window viewport of
1000px wide (screen resolution)
E.g.:
http://test.csswg.org/suites/css2.1/20110323/html4/vertical-align-121.htm
15-
Use description wisely
**********************
http://test.csswg.org/suites/css2.1/20110323/html4/cursor-014.htm
Test passes if a cursor that indicates westward movement appears when
...
(better) Test passes if a double-headed horizontal arrow appears when
...
16-
Avoid technical terms, otherwise describe them
**********************************************
(bad) "You should see a 100x100 green square below."
http://test.csswg.org/suites/css2.1/20110111/html4/clipping-002.htm
(bad) "a 10em box"
(bad) "multi-bit alpha channel"
http://test.csswg.org/suites/css2.1/20110323/html4/background-alpha-001.htm
17-
Graphical layout of objects to compare
**************************************
If measurements are performed by comparing with a referring
{stripe, bar, square, rectangle, line}, then best is to align
horizontallly
the tested geometrical shapes if their height is the object of the test
and
to align vertically the tested geometrical shapes if their width is the
object of the test.
The general idea is to arrange graphical layout of objects in a way that
eases the task of the tester.
18-
Reuse consistently CSS 2.1 terminology
**************************************
In the text assert and in title, reuse the spec terminology, vocabulary
and do not add new, unknown terminology.
(bad) Shrink-to-fit height
http://test.csswg.org/suites/css2.1/20110323/html4/absolute-non-replaced-height-002.htm
(good) Height is based on content.
"shrink-wraps", "shrink-wrapping" are not defined anywhere in the spec.
"ancestor" is preferable/better to "ascendant" because the spec uses the
word "ancestor".
"induce a style"
http://test.csswg.org/suites/css2.1/20110323/html4/table-column-rendering-004.htm
19-
Use meaningful, helpful, semantical id or class attributes
**********************************************************
Bad, not recommendable:
(bad) class="class1"
(bad) class="class2"
(bad) class="one"
(bad) class="two"
(bad) class="a"
(bad) class="b"
(bad) id="div1"
(bad) id="div2"
(bad) id="span1"
(bad) id="span2"
(bad) id="one"
(bad) id="two"
(bad) id="a"
(bad) id="b"
id="test"
function test()
Better, more recommendable:
(good) class="abs-pos-parent"
(good) id="rel-pos-grand-parent"
(good) id="containing-block"
(good) class="blue-float-left"
(good) id="red-overlapped"
(good) id="green-overlapping"
(good) id="reference"
(good) id="control"
(good) id="clearing"
(good) id="following-sibling"
(good) class="adjacent-sibling"
Good identifiers are those which are
- descriptive with regards to the working logic of the testcase,
- descriptive of the design logic of the testcase or
- descriptive of the respective position in the containment hierarchy or
- descriptive of the respective position in the positioning hierarchy.
Identifiers should describe the role of an element inside the building
logic of a testcase.
20-
Include calculations and/or explanations into comments of the testcase
***********************************************************************
Eg.
http://test.csswg.org/suites/css2.1/20110323/html4/position-absolute-percentage-inherit-001.htm
http://test.csswg.org/suites/css2.1/20110323/html4/margin-collapse-clear-013.htm
21- Try to make the most important pass/fail condition of a testcase the
most
important visual aspect of the page and the only/sole visual aspect of
the page
22-
Sufficient background/foreground color contrast/brightness
**********************************************************
Some tests inadvertedly, awkwardly use 2 dark colors. Or they write with
a
dark color inside another dark color area.
eg (bad)
http://test.csswg.org/suites/css2.1/20110323/html4/column-border-002.htm
eg (bad)
http://test.csswg.org/suites/css2.1/20110323/html4/first-line-inherit-002.htm
23-
Graphically arrange layout to help tester
*****************************************
The testcase should be built in a way that, when it fails, it is
graphically
clear and obvious that it fails:
In
http://test.csswg.org/suites/css2.1/20110323/html4/block-non-replaced-width-008.htm
http://test.csswg.org/suites/css2.1/nightly-unstable/html4/column-visibility-004.htm
the amount of red is equal to the amount of green. So, in case of a test
failure, the tester will easily notice the failure.
(bad)
http://test.csswg.org/suites/css2.1/20110323/html4/max-height-017.htm
An UA which does not implement max-height is going to show 2 squares
whose
differences of height is going to be 1px only: for many humans, a hardly
noticeable difference. A machine running software would mark this as
fail
though.
24-
Spelling
********
descendent (eg descendent-selector-00? testcases), PREREQUSITE,
strips (table-borders-002 and 004),
25-
Containing block is not a CSS box
*********************************
The concept of containing block is for positioning; a containing block
is not a CSS box. A containing block is nothing more than a rectangle
formed by (see section 10.1 as there are more than 1 answer) which has a
width, a height, a top edge, a left edge, a bottom edge and a right
edge. A containing block only serves to position positioned elements.
So, "padding of containing block" and "content edge of containing block"
are out of spec terms and should not be used.
26-
Title versus text assert coherence
**********************************
The title of a testcase should never contradict or oppose the assert
text. Coherence, consistence, clarity at all times.
There are *_hundreds_* of testcases where this happens. The title says -
example given
"Fixed position elements impact on later siblings"
but the assert says
"Fixed positioned elements do not impact later siblings."
http://test.csswg.org/suites/css2.1/20110323/html4/position-absolute-001.htm
The thing is that even in the listing of testcases, users will more
likely see the title rather than the assert because of smaller font-size
used for the assert. In any case, a blatant contradiction like this is
never going to help users to understand CSS 2.1. Some people think and
say that CSS 2.1 is tough to read, hard to understand, without enough
examples, etc... and now they could add that the CSS 2.1 test suite does
not help as much as it could... by being consistently clear at all
times.
The title of a testcase should probably only lists the most important
properties or concept involved.
eg:
<title>CSS Test: border-collapse: collapse - adjacent cells with same
border-style and border-width</title>
or
<title>CSS Test: border conflict element - adjacent cells with same
border-style and border-width</title>
27-
Reusing the same rendered layout
********************************
When creating a serie of testcases, try to use and reuse the same
rendered layout. It not only helps the tester but it can reduce the
number
of reftests needed to create later.
Bad:
Stack of 5 colored bands
------------------------
http://test.csswg.org/suites/css2.1/20110323/html4/margin-collapse-106.htm,
112: a stack of 5 bands which are from top to bottom: orange, yellow,
orange, lime, orange
http://test.csswg.org/suites/css2.1/20110323/html4/margin-collapse-110.htm,
111: a stack of 5 bands which are from top to bottom: orange, lime,
orange, yellow, orange
http://test.csswg.org/suites/css2.1/20110323/html4/margin-collapse-114.htm:
a stack of 5 bands which are from top to bottom: yellow, orange, aqua,
lime, orange
Good:
Stack of 3 colored squares
--------------------------
http://test.csswg.org/suites/css2.1/nightly-unstable/html4/z-index-stack-001.htm
http://test.csswg.org/suites/css2.1/nightly-unstable/html4/z-index-stack-002.htm
http://test.csswg.org/suites/css2.1/nightly-unstable/html4/z-index-stack-003.htm
(albeit dimensions need to be uniformized to 100px by 100px)
28-
Font-size
*********
Try to make test use a reasonable font-size: minimum 16px, recommended
20px with a 30px leading. eg font: 20px/1.5 serif;
eg.
(less recommendable)
http://test.csswg.org/suites/css2.1/nightly-unstable/html4/border-padding-bleed-001.htm
(less recommendable)
http://test.csswg.org/suites/css2.1/nightly-unstable/html4/position-absolute-percentage-inherit-001.htm
(better, more recommendable)
http://test.csswg.org/suites/css2.1/nightly-unstable/html4/vertical-align-117a.htm
(better, more recommendable)
http://test.csswg.org/suites/css2.1/nightly-unstable/html4/separated-border-model-007.htm
29-
Consistency of vocabulary, terminology, expressions used, shape
descriptors
***************************************************************************
box is [color name]
http://test.csswg.org/suites/css2.1/20110323/html4/background-image-cover-001.htm
background of [text] is [color name]
http://test.csswg.org/suites/css2.1/20110323/html4/background-transparency-001.htm
At all times, you want, you aim for a test where pass/fail condition
sentence is
unequivocal, disambiguated, tries to establish a 1 to 1 (signifier to
signified) relation.
30-
Unneeded, unnecessary, extraneous declarations
**********************************************
width: 0
--------
If an element applies shrink-to-fit width (like a table cell), then
setting its
width to 0 is unneeded and unnecessary.
http://test.csswg.org/suites/css2.1/20110323/html4/border-right-color-applies-to-001.htm
http://test.csswg.org/suites/css2.1/20110323/html4/border-right-color-applies-to-004.htm
An empty cell will not under normal circumstances require that its box
occupy/creates an horizontal space for its empty content. An empty cell
that has 'width: auto' will use/occupy/generate a zero width box, a
content box of 0 width.
table-layout: fixed
-------------------
In order to trigger the fixed table layout algorithm and to get some
kind of layout effect out of it, some kind of rendering effect out of
it, the table width or at least one column width or at least one table
cell width in first row has to be specified.
If none of those 3 is specified, then 'table-layout: fixed' does not and
can not have any useful or meaningful rendering effect.
http://test.csswg.org/suites/css2.1/20110323/html4/border-left-width-applies-to-001.htm
http://test.csswg.org/suites/css2.1/20110323/html4/border-left-width-applies-to-004.htm
http://test.csswg.org/suites/css2.1/20110323/html4/border-left-width-applies-to-006.htm
etc.
margin-left: 1in
----------------
http://test.csswg.org/suites/css2.1/20110323/html4/border-left-width-applies-to-001.htm
http://test.csswg.org/suites/css2.1/20110323/html4/border-left-width-applies-to-002.htm
http://test.csswg.org/suites/css2.1/20110323/html4/border-left-width-applies-to-003.htm
http://test.csswg.org/suites/css2.1/20110323/html4/border-left-width-applies-to-004.htm
http://test.csswg.org/suites/css2.1/20110323/html4/border-left-width-applies-to-005.htm
http://test.csswg.org/suites/css2.1/20110323/html4/border-left-width-applies-to-006.htm
http://test.csswg.org/suites/css2.1/20110323/html4/border-left-width-applies-to-007.htm
http://test.csswg.org/suites/css2.1/20110323/html4/border-left-width-applies-to-013.htm
http://test.csswg.org/suites/css2.1/20110323/html4/border-left-width-applies-to-014.htm
In the 'border-collapse: collapse' model,
"
The left border width of the table is half of the first cell's collapsed
left border, and the right border width of the table is half of the last
cell's collapsed right border. If subsequent rows have larger collapsed
left and right borders, then any excess spills into the margin area of
the table.
"
http://www.w3.org/TR/CSS21/tables.html#collapsing-borders
There is no subsequent rows in those tests, so no possible excess
spilling. So, I see no reason to set a margin-left to 1in for these
tests.
Where exactly the left border of a table starts in comparison to where
its content box starts is a distinct and separate issue which does not
and should not affect where the black border starts being painted in all
those tests.
http://lists.w3.org/Archives/Public/public-css-testsuite/2012Jun/0000.html
height: 0
---------
http://test.csswg.org/suites/css2.1/20110323/html4/border-bottom-width-061.htm
http://test.csswg.org/suites/css2.1/20110323/html4/border-bottom-width-062.htm
http://test.csswg.org/suites/css2.1/20110323/html4/border-top-width-061.htm
http://test.csswg.org/suites/css2.1/20110323/html4/border-top-width-062.htm
http://test.csswg.org/suites/css2.1/20110323/html4/border-width-001.htm
http://test.csswg.org/suites/css2.1/20110323/html4/border-width-002.htm
http://test.csswg.org/suites/css2.1/20110323/html4/border-width-003.htm
http://test.csswg.org/suites/css2.1/20110323/html4/border-width-004.htm
http://test.csswg.org/suites/css2.1/20110323/html4/border-width-005.htm
http://test.csswg.org/suites/css2.1/20110323/html4/border-width-006.htm
http://test.csswg.org/suites/css2.1/20110323/html4/border-width-007.htm
Under normal circumstances, an empty div does not require height: 0.
Under
normal circumstances, an div with 'height: auto' which does not have any
content
nor children will occupy/generate a height of 0.
height: 1in
-----------
http://test.csswg.org/suites/css2.1/20110323/html4/border-top-width-092.htm
http://test.csswg.org/suites/css2.1/20110323/html4/border-top-width-093.htm
http://test.csswg.org/suites/css2.1/20110323/html4/border-top-width-094.htm
Removed because pointless, irrelevant to the tests.
width: 1in set to an inline non-replaced
----------------------------------------
http://test.csswg.org/suites/css2.1/20110323/html4/box-generation-001.htm
http://test.csswg.org/suites/css2.1/20110323/html4/box-generation-002.htm
Setting a width to an inline non-replaced element has no effect, has no
rendering or layout effect.
border-collapse: collapse with border-spacing: 0
------------------------------------------------
border-spacing does not apply in border-collapse: collapse model. So,
just border-collapse: collapse is and should be sufficient.
* {margin: 0}
-------------
This is not suitable, not needed and not recommendable in tests.
31-
1px difference
**************
div#test {max-height: 72pt; height: 73pt;}
div#control {height: 72pt;}
(...)
<p>Test passes if the two boxes below are the same height.</p>
Example taken from
http://test.csswg.org/shepherd/testcase/max-height-017/#comment-c3e1d615f21e
http://test.csswg.org/suites/css2.1/20110323/html4/max-height-017.htm
32-
Single character testcases
**************************
Testcases which involve comparison of 2 characters or which involve a
single character should be using a minimum font-size of 20px or even
bigger (say 32px or 64px).
Some test pass condition involve the color of a single character.
E.g.:
http://test.csswg.org/suites/css2.1/20110323/html4/first-letter-selector-001.htm
and there are many first-letter testcases
E.g.:
http://test.csswg.org/suites/css2.1/20110323/html4/character-representation-001.htm
should have the 2 characters at least 32px.
Anything that help testers, reduce condition of mistakes, speed their
tasks, etc. is better.
33-
Miscellaneous
*************
- Some test create a vertical scrollbar (not sure why or if this is part
of test). E.g.:
http://test.csswg.org/suites/css2.1/20110323/html4/floats-rule3-outside-left-001.htm
- File numbering: should always start at 001, not at 000: that would
make
counting much easier. E.g.: property-variable-001.xht has to be the
first
testcase of a serie of testcases with that same base filename.
- Tests with a font-size under 16px. Unless the test goals involves,
requires testing small font-size, all tests should be using the browser
default font-size of 16px.
34-
Better color descriptors
************************
Lime or bright green or light green is not green.
eg
http://test.csswg.org/suites/css2.1/20110323/html4/inlines-020.htm
Navy blue or navy is not blue.
http://test.csswg.org/suites/css2.1/20110323/html4/blocks-011.htm
http://test.csswg.org/suites/css2.1/20110323/html4/blocks-012.htm
35-
Standardized color for the pass/fail conditions sentence
********************************************************
Some Ian 'Hixie' Hickson tests use unneedlessly blue navy for the
pass/fail conditions sentence.
36-
Too long
********
Filename too long:
http://test.csswg.org/suites/css2.1/20110323/html4/position-absolute-percentage-inherit-001.htm
Test is maybe too long:
http://test.csswg.org/suites/css2.1/nightly-unstable/html4/font-family-rule-004a.htm
That test should maybe be split... on the other hand, the font to be
loaded will be good for all those 11 sub-tests.
37-
Short but clear pass-fail conditions sentence
*********************************************
Ideally, you want a short but clear pass-fail conditions sentence: the
shorter, the better. Less reading; less thinking, less prone to
interpretation.
http://www.gtalbot.org/BrowserBugsSection/css21testsuite/cursor-025.html
had 146 characters:
"
Hover the pointing device inside the rectangle. Test passes if the
pointing device becomes a small bright green square with the word "PASS"
in it.
"
and it was changed to:
"
Test passes if the cursor becomes [PASS] when inside the rectangle.
"
which has 60 characters.
38-
Testing 'inherit' keyword on properties that inherit by default
*******************************************************************
Tests that check 'inherit' keyword on properties that inherit by default
should be removed from test suites. Testing with 'inherit' keyword
should only be done on properties that do not inherit by default.
So:
http://test.csswg.org/suites/css2.1/nightly-unstable/html4/list-style-type-016.htm
http://test.csswg.org/suites/css2.1/nightly-unstable/html4/font-size-112.htm
(this one even is affected by a specificity issue)
http://test.csswg.org/suites/css2.1/nightly-unstable/html4/caption-side-003.htm
etc.
would be removed.
39-
An ideal test is ...
********************
An ideal test is one that has a very narrow corridor of success and very
large corridor of failure.
Eg.:
http://test.csswg.org/suites/css-backgrounds-3_dev/nightly-unstable/html4/background-size-009.html
at line 22:
I changed 'height: 100px' to 'height: 50px' to "enlarge" the corridor of
failure.
More info:
http://test.csswg.org/shepherd/testcase/background-size-009/#comment-2f7643da4c58
http://test.csswg.org/suites/css-backgrounds-3_dev/nightly-unstable/html4/background-size-014.html
at line 24:
I changed 'width: 200px' for 'width: 50px' to "enlarge" the corridor of
failure.
More info:
http://test.csswg.org/shepherd/testcase/background-size-014/#comment-f2d2f913b910
An ideal test is one that is very demanding, requiring.
An ideal test is one that checks, verifies one and only one single
aspect of the spec.
Anything that simplifies the task of testers (people taking the tests)
is, as a general rule, welcomed, better, more desirable.
Maintaining a moderate skepticism, vigilance toward browser
manufacturers and implementors is a sane and cautious test
creation/review attitude to adopt; "Trust no browser!" means to not
develop tests with browsers, not develop tests alongside browsers but to
develop tests thanks to and starting with only or mostly with your
understanding of the spec.
--------------
Friday August 21st 2015
ref-black-96px-square-no-red : referenced by 12 tests
ref-filled-black-96px-square : referenced by 180 tests
ref-filled-green-100px-square : referenced by 319 tests (5 changesets! +
40 more if at-supports-001-ref.html would be modified a bit.)
ref-if-there-is-no-red : referenced by 289 tests (2 changesets!)
ref-no-vert-space-between : referenced by 188 tests
ref-nothing-below : referenced by 22 tests
ref-this-text-should-be-green : referenced by 6 tests
========================================================
1046 tests
Friday August 21st 2015
--
Writing Tests
http://web-platform-tests.org/writing-tests/
General Test Guidelines
http://web-platform-tests.org/writing-tests/general-guidelines.html
Test Review Checklist
http://web-platform-tests.org/reviewing-tests/checklist.html
Test Templates
http://web-platform-tests.org/appendix/test-templates.html
Writing Reftests
http://web-platform-tests.org/writing-tests/reftests.html
Received on Monday, 11 June 2018 22:27:30 UTC