- From: Gérard Talbot <css21testsuite@gtalbot.org>
- Date: Thu, 29 Oct 2015 15:02:39 -0400
- To: Geoffrey Sneddon <me@gsnedders.com>
- Cc: Public CSS Test suite mailing list <public-css-testsuite@w3.org>
Le 2015-10-29 02:05, Geoffrey Sneddon a écrit : > On Thu, Oct 29, 2015 at 6:09 AM, Gérard Talbot > <css21testsuite@gtalbot.org> > wrote: > >> Le 2015-10-27 03:31, Geoffrey Sneddon a écrit : [snipped] >> I'm all for a discussion on metadata and documentation ... but what >> about >> bad or wrong tests? >> I wish >> a) incorrect tests, >> b) imprecise tests, >> c) tests that can not fail (unreliable tests, non-trustworthy tests) >> and >> d) tests that do not check what they claim to be checking >> would be removed or rehabilitated or dealt with to start with. I've >> been >> asking for this in the last 4 years (june 28th 2011 and even before >> that) >> and it still has not been dealt with. And I'm not talking about a few >> dozen >> tests here... > > > a) and b) are quite easily found if browsers are actually running them quite easily found? I have doubts... > and > are able to contribute fixes back upstream (which is another problem > we've > had for years). [snipped] For CSS2.1 tests: a) 65 CSS2.1 tests with Whiteboard NeedsWork=Incorrect http://test.csswg.org/shepherd/search/testcase/spec/css21/status/issue/whiteboard/Incorrect/ b) 74 CSS2.1 tests with Whiteboard NeedsWork=Precision http://test.csswg.org/shepherd/search/testcase/spec/css21/status/issue/whiteboard/Precision/ >> Personally, I think about 30% to 40% of all existing tests could be >> re-engineered so that they would be associated with already available, >> already created and very frequently reused reference files. When I >> create a >> test, I always try to do this myself. That way, >> a) I no longer have to think about creating a reference file, >> b) this reduces server load when "doing" a test suite with the test >> harness and >> c) this reduces the growth of N reference files to be referenced >> >> Examples given: >> >> ref-if-there-is-no-red : referenced by 290 tests (2 changesets!) >> >> http://test.csswg.org/shepherd/search/reference/name/ref-if-there-is-no-red/ >> http://test.csswg.org/source/css21/reference/ref-if-there-is-no-red.xht >> >> ref-this-text-should-be-green : referenced by 43 tests >> test.csswg.org/shepherd/reference/ref-this-text-should-be-green/ >> >> http://test.csswg.org/source/css21/reference/ref-this-text-should-be-green.xht >> >> So, why re-create 2 reference files that already exist? > > > Because if there's 300 tests with "there should be no red below" and > 200 > with "there should be no red", it can easily be quite hard to remove > the > word "below", because removing the word can cause later content to > reflow > and the test to then fail. [snipped] Geoffrey, I tried to understand what you're saying and just could not. [Addendum: after more thinking, now I remember 1 test where what you described could *maybe* happen.] Eg. 37 tests in http://test.csswg.org/source/css-conditional-3/ are using, associating with the reference file http://test.csswg.org/source/css-conditional-3/at-supports-001-ref.html when it would be *_very easy_* to adapt those 37 tests to use, to link to http://test.csswg.org/source/css21/reference/ref-filled-green-100px-square.xht I probably could do this *_in less than_* 10 min. thanks to advanced search and replace. The thing is: the current (and past) documentation are not encouraging test authors to reuse already created and available reference files. Gérard -- Test Format Guidelines http://testthewebforward.org/docs/test-format-guidelines.html Test Style Guidelines http://testthewebforward.org/docs/test-style-guidelines.html Test Templates http://testthewebforward.org/docs/test-templates.html CSS Naming Guidelines http://testthewebforward.org/docs/css-naming.html Test Review Checklist http://testthewebforward.org/docs/review-checklist.html CSS Metadata http://testthewebforward.org/docs/css-metadata.html
Received on Thursday, 29 October 2015 19:03:16 UTC