Re: Format guidelines for tests (particularly HTML5)

On Apr 16, 2012, at 10:25 AM, Dirk Schulze wrote:

> Hi,
> 
> On Apr 16, 2012, at 10:14 AM, Linss, Peter wrote:
> 
>> 
>> On Apr 16, 2012, at 4:09 AM, Aryeh Gregor wrote:
>> 
>>> Dirk Schulze submitted a bunch of feedback on my CSS Transforms test
>>> submissions, mostly stylistic:
>>> <http://test.csswg.org/shepherd/testcase/abspos-1b#event-4>.  I'd like
>>> to make sure that it's agreed upon and written down somewhere.  So
>>> here are some issues:
>>> 
>>> 1) Tests in HTML format are accepted, but there's no template for
>>> them.  The test format page <http://wiki.csswg.org/test/format> says
>>> XHTML is preferred.  I suggest that it be changed to say that either
>>> XHTML or HTML5 is preferred, and the following template be added:
>>> 
>>> <!DOCTYPE html>
>>> <html>
>>> <head>
>>> <title>CSS Test: DESCRIPTION OF TEST</title>
>>> <link rel="author" title="NAME_OF_AUTHOR" href="mailto:EMAIL OR
>>> http://CONTACT_PAGE"/>
>>> <link rel="help" href="RELEVANT_SPEC_SECTION"/>
>>> <meta name="flags" content="TOKENS" />
>>> <meta name="assert" content="TEST ASSERTION"/>
>>> <style>
>>> CSS FOR TEST
>>> </style>
>>> </head>
>>> <body>
>>> CONTENT OF TEST
>>> </body>
>>> </html>
>> 
>> Look reasonable to me. Just remove the trailing '/' on the link and meta elements.
>> 
>>> 
>>> Note that I removed the CDATA, and the type="text/css" (which is
>>> unnecessary in HTML5).  I personally don't think <html>/<head>/<body>
>>> should be used unless necessary, but it seems most people disagree
>>> with me.  Given that, I'm okay with this template.
>>> 
>>> 2) Additionally, the guidelines should say that quotes on attribute
>>> values shouldn't be omitted for HTML5.  Again, I prefer omitting
>>> quotes for brevity, but it seems most people disagree.
>> 
>> One of those situations where the extra safety wins over brevity.
>> 
>>> 
>>> 3) It should be more specific about indentation.  The template uses
>>> one space for indentation; the tests I submitted use two spaces
>>> (Mozilla's convention); Dirk prefers four.  Do we have a preference?
>> 
>> I really don't want to get into yet another indentation thread, I'm sure there are some that prefer tabs as well. I think indentation is valuable, I don't think mandating a specific style is worth it. Just keep it readable and if you edit someone else's test, try to stay consistent.
> I just want to clarify that we should use indentations. Ideally we use the same indentation on all tests in the same style. If tabs, spaces (and how many whitespaces) are better, is subjective. So I am fine if we at least agree that the current guidelines recommends indentation. And it does already.

Yes, indentation should be used to make the tests more readable.

> 
>> 
>>> 
>>> 4) Dirk said that multiline <title>s cause problems for Shepherd.  If
>>> this is so, should we make sure tests are formatted with one-line
>>> <title>s or fix Shepherd?  (I usually break lines at 79 chars, which
>>> is why I wrapped some of my <title>s to start with.)
>> 
>> Note that Shepherd uses the test suite build code to extract metadata from the tests, so this looks like it's actually a bug in the build code (or possibly the HTML5 parser it uses). I'll look in to it. There shouldn't be a restriction on the number of lines the title takes up, but it should be kept brief.
> How do we interact in the meantime? Assume that this will be fixed in the future?

Assume it will be fixed.

> 
>> 
>>> 
>>> 5) The template's <title> starts with "CSS Test: ".  I'm guessing this
>>> was written when all the tests were for CSS 2.1.  I wrote my transform
>>> tests starting with "CSS Test (Transforms): " -- should we adopt a
>>> convention like this for non-CSS 2.1 tests?  Maybe it should be "CSS
>>> Test: Transforms - " or something else?  It looks like Shepherd
>>> currently eats everything up to the first colon, so the "(Transforms)"
>>> part is vanishing on Shepherd.
>> 
>> Again, this processing is done by the build code, not Shepherd. It actually uses a regex to strip any preceding text that matches "*Test*:". 
>> 
>> And yes, that guideline was initially written for the 2.1 test suite (as they all were). I believe the intention here is that the title be descriptive of the page when viewed standalone. Since our tools already provide the context, we strip the prefix in our display of the titles.
>> 
>> I think it's fine to title your tests "CSS Test (Transforms): …" or perhaps "CSS Transforms Test: …". The title convention should be consistent within a test suite and should not include spec level numbers (as the test may get re-used for a later version of the spec).
> I would vote for "CSS Transforms/Regions/Units and Values Tests". Nevertheless, if I understand this correctly, there is no requirement for any prefix, as long as it contains "Test" a ":".

Correct. The prefix itself is not strictly mandatory, but is good practice. If the prefix exists, it must contain the word "Test" and end with a colon. The prefix will remain present in the test page, but will be stripped in the test suite index pages, Shepherd and the harness.

Received on Monday, 16 April 2012 18:23:38 UTC