Re: CSS 2.1 test suite feedback: organizational

On Sep 27, 2010, at 7:06 pm, fantasai wrote:

> On 09/27/2010 08:15 AM, Simon Fraser wrote:
>> On Sep 27, 2010, at 2:02 am, fantasai wrote:
>>> On 09/26/2010 11:27 PM, Simon Fraser wrote:
>>>> I've been through about 60% of the HTML4 tests in the 20100917 suite, and have some feedback.
>>>> For pure cleanliness reasons, I suggest that the files in the html4 and xhtml1
>>>> directories are broken into test files and non-test files. I suggest a hierarchy
>>>> like the following:
>>>> [html|xhtml1]/
>>>>            toc.html
>>>>            chapters/
>>>>                     chapter-1.[htm|xht]
>>>>                     ...
>>>>            tests/
>>> This should be doable. I am wondering, however, why the chapter tocs are not in
>>> the main directory?
>> They could be; that would be fine.
> Would it make sense to separate out reftests and selftests into their own
> directories?
> Note we also have tests that are both.

I don't think that's necessary.

>>>> indices: index of the test in the chapters it appears; comma-separated list with
>>>> the same length as chapters (e.g. "134, 12,10").
>>> Hmm, this might be difficult. Why do you need this information?
>> My test harness is currently scraping the chapter files to present the testcases
>> in the same order they appear there, since seems to have a random
>> sort order.
>> It's useful to present tests in order, so that similar tests are grouped, lowering
>> the amount of mental effort for each test.
>> Perhaps I can just group based on the links info, and the sort by filename
>> within those categories? Is that how the chapter files are organized?
> The best thing to do would be to use the order in the implementation report
> template. That one is sorted by filename first, format name second. This
> is probably the best way to sort it: it's a stable order, very simple to
> determine, groups tests belonging to a series together, and does not have
> duplicates. It will jump around chapters a bit, but tests on a single topic
> tend to belong to the same series.

One reason I used the chapter groupings in my harness (details to come soon)
are that they give a tester targets to work for, and for implementors, it's useful to know
what the weak areas are.

>> Any test that requires special action to test (and therefore cannot be run automatically)
>> should have a flag to indicate this. Are there any other prerequisites of this nature
>> which are not flagged yet?
> I am not sure. I think gsnedders added a flag for 'animated'. And
> as I mentioned there's already one for 'interact'.

I'll let you know if I find any more while testing.


Received on Tuesday, 28 September 2010 03:25:09 UTC