Re: CSS 2.1 test suite feedback: organizational

On Sep 27, 2010, at 2:02 am, fantasai wrote:

> On 09/26/2010 11:27 PM, Simon Fraser wrote:
>> I've been through about 60% of the HTML4 tests in the 20100917 suite, and have some feedback.
>> 
>> For pure cleanliness reasons, I suggest that the files in the html4 and xhtml1
>> directories are broken into test files and non-test files. I suggest a hierarchy
>> like the following:
>> 
>> [html|xhtml1]/
>>               toc.html
>>               chapters/
>>                        chapter-1.[htm|xht]
>>                        ...
>>               tests/
> 
> This should be doable. I am wondering, however, why the chapter tocs are not in
> the main directory?

They could be; that would be fine.

>> It would be very helpful if testinfo.data file had the following additional columns,
>> to allow a test harness to categorize and present tests in the same order that they
>> are presented in chapters:
>> 
>> chapters: comma-separated list of chapters that the test is applicable to (e.g. "3, 4, E").
> 
> This can be derived from the links column.

Ah, good to know.

> 
>> indices: index of the test in the chapters it appears; comma-separated list with
>> the same length as chapters (e.g. "134, 12,10").
> 
> Hmm, this might be difficult. Why do you need this information?

My test harness is currently scraping the chapter files to present the testcases
in the same order they appear there, since testinfo.data seems to have a random
sort order.

It's useful to present tests in order, so that similar tests are grouped, lowering
the amount of mental effort for each test.

Perhaps I can just group based on the links info, and the sort by filename 
within those categories? Is that how the chapter files are organized?

> 
>> It would also be helpful to know the prerequisites for each test:
>> * whether they should be viewed with a paged media type
> 
> This is in the metadata as the 'paged' flag.
> 
>> * whether they must be served over http
> 
> This is in the metadata as the 'http' flag.
> 
>> * whether they require user interaction
> 
> This is in the metadata as the 'interact' flag.

Good to know, thanks.

> 
>> * whether they require a user CSS file
> 
> This is not flagged specially. If you make a list of such tests,
> I can add flags for the next release.

I think this flag would be useful.
> 
> The flags are defined here:
>  http://wiki.csswg.org/test/css2.1/format#requirement-flags
> and should appear both in the test and in the testinfo.data file.
> If a particular test is not flagged correctly, then it's an error
> in the test and should be reported as such.

 Any test that requires special action to test (and therefore cannot be run automatically)
should have a flag to indicate this. Are there any other prerequisites of this nature
which are not flagged yet?

> 
>> Can the "dinamic-pseudo-classes" be renamed to fix the spelling error?
> 
> Thought I already did that. :/ I'll go fix that now...

Thanks
Simon

Received on Monday, 27 September 2010 15:16:31 UTC