W3C home > Mailing lists > Public > public-css-testsuite@w3.org > September 2010

Re: CSS 2.1 test suite feedback: organizational

From: Simon Fraser <smfr@me.com>
Date: Mon, 27 Sep 2010 20:24:32 -0700
Cc: public-css-testsuite@w3.org
Message-id: <12E8A4E3-6F42-4A5E-AFC5-19361488994F@me.com>
To: fantasai <fantasai.lists@inkedblade.net>
On Sep 27, 2010, at 7:06 pm, fantasai wrote:

> On 09/27/2010 08:15 AM, Simon Fraser wrote:
>> On Sep 27, 2010, at 2:02 am, fantasai wrote:
>>> On 09/26/2010 11:27 PM, Simon Fraser wrote:
>>>> I've been through about 60% of the HTML4 tests in the 20100917 suite, and have some feedback.
>>>> 
>>>> For pure cleanliness reasons, I suggest that the files in the html4 and xhtml1
>>>> directories are broken into test files and non-test files. I suggest a hierarchy
>>>> like the following:
>>>> 
>>>> [html|xhtml1]/
>>>>            toc.html
>>>>            chapters/
>>>>                     chapter-1.[htm|xht]
>>>>                     ...
>>>>            tests/
>>> 
>>> This should be doable. I am wondering, however, why the chapter tocs are not in
>>> the main directory?
>> 
>> They could be; that would be fine.
> 
> Would it make sense to separate out reftests and selftests into their own
> directories?
> 
> Note we also have tests that are both.

I don't think that's necessary.

>>>> indices: index of the test in the chapters it appears; comma-separated list with
>>>> the same length as chapters (e.g. "134, 12,10").
>>> 
>>> Hmm, this might be difficult. Why do you need this information?
>> 
>> My test harness is currently scraping the chapter files to present the testcases
>> in the same order they appear there, since testinfo.data seems to have a random
>> sort order.
>> 
>> It's useful to present tests in order, so that similar tests are grouped, lowering
>> the amount of mental effort for each test.
>> 
>> Perhaps I can just group based on the links info, and the sort by filename
>> within those categories? Is that how the chapter files are organized?
> 
> The best thing to do would be to use the order in the implementation report
> template. That one is sorted by filename first, format name second. This
> is probably the best way to sort it: it's a stable order, very simple to
> determine, groups tests belonging to a series together, and does not have
> duplicates. It will jump around chapters a bit, but tests on a single topic
> tend to belong to the same series.

One reason I used the chapter groupings in my harness (details to come soon)
are that they give a tester targets to work for, and for implementors, it's useful to know
what the weak areas are.

>> Any test that requires special action to test (and therefore cannot be run automatically)
>> should have a flag to indicate this. Are there any other prerequisites of this nature
>> which are not flagged yet?
> 
> I am not sure. I think gsnedders added a flag for 'animated'. And
> as I mentioned there's already one for 'interact'.

I'll let you know if I find any more while testing.

Simon
Received on Tuesday, 28 September 2010 03:25:09 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Tuesday, 28 September 2010 03:25:16 GMT