W3C home > Mailing lists > Public > public-css-testsuite@w3.org > October 2015

Re: Simplifying metadata

From: Gérard Talbot <css21testsuite@gtalbot.org>
Date: Thu, 29 Oct 2015 15:02:39 -0400
To: Geoffrey Sneddon <me@gsnedders.com>
Cc: Public CSS Test suite mailing list <public-css-testsuite@w3.org>
Message-ID: <3cd3b9384a8c2eb61eeb34097ca54b03@gtalbot.org>
Le 2015-10-29 02:05, Geoffrey Sneddon a écrit :
> On Thu, Oct 29, 2015 at 6:09 AM, Gérard Talbot 
> <css21testsuite@gtalbot.org>
> wrote:
>> Le 2015-10-27 03:31, Geoffrey Sneddon a écrit :


>> I'm all for a discussion on metadata and documentation ... but what 
>> about
>> bad or wrong tests?
>> I wish
>> a) incorrect tests,
>> b) imprecise tests,
>> c) tests that can not fail (unreliable tests, non-trustworthy tests) 
>> and
>> d) tests that do not check what they claim to be checking
>> would be removed or rehabilitated or dealt with to start with. I've 
>> been
>> asking for this in the last 4 years (june 28th 2011 and even before 
>> that)
>> and it still has not been dealt with. And I'm not talking about a few 
>> dozen
>> tests here...
> a) and b) are quite easily found if browsers are actually running them

quite easily found? I have doubts...

> and
> are able to contribute fixes back upstream (which is another problem 
> we've
> had for years).


For CSS2.1 tests:

a) 65 CSS2.1 tests with Whiteboard NeedsWork=Incorrect

b) 74 CSS2.1 tests with Whiteboard NeedsWork=Precision

>> Personally, I think about 30% to 40% of all existing tests could be
>> re-engineered so that they would be associated with already available,
>> already created and very frequently reused reference files. When I 
>> create a
>> test, I always try to do this myself. That way,
>> a) I no longer have to think about creating a reference file,
>> b) this reduces server load when "doing" a test suite with the test
>> harness and
>> c) this reduces the growth of N reference files to be referenced
>> Examples given:
>> ref-if-there-is-no-red  : referenced by 290 tests (2 changesets!)
>> http://test.csswg.org/shepherd/search/reference/name/ref-if-there-is-no-red/
>> http://test.csswg.org/source/css21/reference/ref-if-there-is-no-red.xht
>> ref-this-text-should-be-green : referenced by 43 tests
>> test.csswg.org/shepherd/reference/ref-this-text-should-be-green/
>> http://test.csswg.org/source/css21/reference/ref-this-text-should-be-green.xht
>> So, why re-create 2 reference files that already exist?
> Because if there's 300 tests with "there should be no red below" and 
> 200
> with "there should be no red", it can easily be quite hard to remove 
> the
> word "below", because removing the word can cause later content to 
> reflow
> and the test to then fail.


Geoffrey, I tried to understand what you're saying and just could not. 
[Addendum: after more thinking, now I remember 1 test where what you 
described could *maybe* happen.]


37 tests in
are using, associating with the reference file
when it would be *_very easy_* to adapt those 37 tests to use, to link 

I probably could do this *_in less than_* 10 min. thanks to advanced 
search and replace.

The thing is: the current (and past) documentation are not encouraging 
test authors to reuse already created and available reference files.

Test Format Guidelines

Test Style Guidelines

Test Templates

CSS Naming Guidelines

Test Review Checklist

CSS Metadata
Received on Thursday, 29 October 2015 19:03:16 UTC

This archive was generated by hypermail 2.4.0 : Friday, 20 January 2023 19:58:21 UTC