W3C home > Mailing lists > Public > public-css-testsuite@w3.org > June 2015

[Test harness] Background-color code of row for test result and conflicting test results

From: Gérard Talbot <css21testsuite@gtalbot.org>
Date: Fri, 05 Jun 2015 14:03:15 -0400
To: Peter Linss <peter.linss@hp.com>
Cc: Public CSS Test suite mailing list <public-css-testsuite@w3.org>
Message-ID: <a056691780299fce446c26f22f063663@gtalbot.org>
Peter,

http://test.csswg.org/harness/results/css-writing-modes-3_dev/grouped/table-column-order-002/

The background-color of the table-column-order-002 test is light green 
indicating that "two or more passes".

But when we look at individual test results:

http://test.csswg.org/harness/details/css-writing-modes-3_dev/table-column-order-002/

we see that a wide majority of test result input from people taking such 
test is "FAIL".

First of all, I was the tester with Firefox Linux x86_64 with source 
"184.160....".

When you do a lot of tests with the test harness, fatigue, tedious 
repetitive looking can certainly make it possible to do 2 or 3 wrong 
input of test results out of 100. I think this is also what happened to 
Koji on 2015-03-21 00:22:00 EDT

http://test.csswg.org/harness/details/css-writing-modes-3_dev/table-column-order-002/

because we can see 2 contradictory test results for the same browser 
(Chrome 41.0.2272.89) on the same day by the same person. The 
possibility of a wrong test input is increased for Chrome browser 
because we have to first click a bookmarklet that will prepend, suffix 
"-webkit-" string in front of writing-mode properties. If we forget 
doing so, then layout of test and test results are wrong.

What I am trying to say here is: when a clear majority of test results 
says "FAIL", I think the row color code should be updated accordingly.

The table-column-order-002 row test result is not unique. I can see 
inappropriate row color code for abs-pos-non-replaced-icb-vlr-003, 
abs-pos-non-replaced-icb-vlr-005, abs-pos-non-replaced-icb-vlr-011, 
abs-pos-non-replaced-icb-vlr-013 tests.

Gérard
-- 
Test Format Guidelines
http://testthewebforward.org/docs/test-format-guidelines.html

Test Style Guidelines
http://testthewebforward.org/docs/test-style-guidelines.html

Test Templates
http://testthewebforward.org/docs/test-templates.html

CSS Naming Guidelines
http://testthewebforward.org/docs/css-naming.html

Test Review Checklist
http://testthewebforward.org/docs/review-checklist.html

CSS Metadata
http://testthewebforward.org/docs/css-metadata.html
Received on Friday, 5 June 2015 18:03:46 UTC

This archive was generated by hypermail 2.4.0 : Friday, 20 January 2023 19:58:21 UTC