Re: Getting started on the Documentation Practices task force

Bryan,

The plan for CSS Regions (that has not been implemented yet) is that we will
consider any ID in the spec to be testable unless the ID is within an
element with an "informative" class attribute. This is an experiment on my
part - I expect other CSS specs may adopt this plan only if my experiment
proves useful.

We arrived at this plan because I found significant pushback on the idea of
adding assertion metadata such as that described at
http://www.w3.org/wiki/Testing/Assertions. Objections were raised that
assertion metadata was too verbose, and that it would be an annoyance to add
and maintain it. Since there are more testable sections to a specification
than non-testable sections, it was deemed easier and more succinct to add
'non-testable' metadata and assume that everything left would require tests.

So the scan for testable IDs is for any ID in the specification that is not
contained in an "informative" class element. I have not yet applied this
approach, but I plan to have the tests contain both a link to an appropriate
section ID and additional links to other relevant IDs (adding paragraph or
span-level IDs as necessary).

Thanks,

Alan

On 1/31/12 6:39 PM, "Bryan Sullivan" <blsaws@gmail.com> wrote:

> Thanks, Peter. I will reference that material.
> 
> A couple of questions:
> 
> I saw in the Regions spec that class="assert must" is used in a few
> places but not many. Other than that I had a hard time seeing any
> specific references to test assertions or tagging of specific
> paragraphs with IDs etc as mentioned in "Each of the Regions tests
> will have metadata that points back one or more ids used in the
> Regions spec. These ids may be heading or dfn elements, or they may be
> ids added at the paragraph or sentence level to indicate a specific
> conformance requirement that must be tested. We should be able to scan
> the spec for ids (ignoring non-testable ids) and find at least one
> test for every testable id in the spec."
> (http://wiki.csswg.org/spec/css3-regions#spec-and-test-metadata). How
> do I know that an ID is testable or scan to determine as noted?
> 
> Are there specific examples (other than the class attribute above)
> that you can point to of assertion markup in the CSS specs? Or is the
> current practice to reference section IDs in the test specs (which is
> what I would call a reverse reference approach, rather than deciding
> up front in the spec what is intended to be testable and providing a
> specific assertion tag to it).
> 
> Thanks for your help.
> 
> On Tue, Jan 31, 2012 at 4:01 PM, Linss, Peter <peter.linss@hp.com> wrote:
>> The CSS WG has a bunch of documentation about our test review process at:
>> http://wiki.csswg.org/test/css2.1/review
>> 
>> Note that this was written before we had our test suite management system
>> online at:
>> http://test.csswg.org/shepherd
>> 
>> Test assertions as part of a spec here:
>> http://wiki.csswg.org/spec/css3-regions#spec-and-test-metadata
>> 
>> Peter
>> 
>> On Jan 31, 2012, at 3:14 PM, Bryan Sullivan wrote:
>> 
>>> Oops, gotta love Gmail.
>>> 
>>> Anyway I was going to continue, that if anyone has input on those
>>> objectives please chime in on this list, and help me develop the set
>>> of info that will help us determine consensus etc. I know there is
>>> some useful prior work and current practices that will also be
>>> referenced, and some analysis done on the pros/cons of them might be
>>> useful. But first spreading the word about how groups go about
>>> developing test assertions and test documentation/assets, and what
>>> methods might be useful to consider for groups who don't have a
>>> specific method for these, is the first step.
>>> 
>>> In addition to the page noted below, some initial data is provided at
>>> http://www.w3.org/wiki/Testing/Assertions. This is a rough first pass
>>> and incomplete, purely in the data collection phase of this.
>>> 
>>> Thanks,
>>> Bryan Sullivan
>>> 
>>> On Tue, Jan 31, 2012 at 3:06 PM, Bryan Sullivan <blsaws@gmail.com> wrote:
>>>> Hi all,
>>>> 
>>>> To get the ball rolling on the "Documentation Practices" task force of
>>>> the Web Testing Interest Group, that I agreed to lead during TPAC, I
>>>> have created a set of wiki pages where the work in progress of the TF
>>>> can be collected. As noted on the wiki page
>>>> (http://www.w3.org/wiki/Testing/Documentation_Practices), the goal of
>>>> this task force is to:
>>>> * determine the current practices for
>>>>        developing test assertions as part of W3C specs
>>>>        processes to review and approve tests
>>>>    broadening consensus on the above
>>>>    promoting adoption of the consensus approach
>>>> 
>>>> --
>>>> Thanks,
>>>> Bryan Sullivan
>>> 
>>> 
>>> 
>>> --
>>> Thanks,
>>> Bryan Sullivan
>>> 
>> 
> 
>

Received on Wednesday, 1 February 2012 04:11:15 UTC