Re: Suggestions for Testing Framework for the masses

On Jun 27, 2012, at 20:13 , Jeanne Spellman wrote:
> On 6/22/2012 5:53 PM, Robin Berjon wrote:
>> On May 24, 2012, at 18:40 , Linss, Peter wrote:
>>>> 1) Specification import:  There needs to be a webUI to import spec data from specs with complex numbering.  The current spec import UI cannot handle any of the WAI guideline-oriented specs, or any spec using a letter-number combination.  Changing the way a spec is formatted - especially a spec with a long history - can become a contentious distraction to a WG.  It's faster to add flexibility to the testing tool, then to get a WG to reach consensus on a new look to their document.  :) Providing options for formats, or simply allowing a webUI import of a data file with the spec info would be helpful.
>>> 
>>> I will soon be re-writing the specification import code to let it identify valid link targets beyond secant headings. When I get to that I'll make sure it can handle the WAI specs as well.
>> 
>> I've been wondering if using the HTML5 outlining algorithm[0] and then using some heuristics to locate the closest ID. I believe that pubrules enforces having an ID on sections anyway so it might just always be there. Using that we could actually just ignore whatever numbering the specification authors may have preferred (which doesn't hurt since I guess people will move to generated content for that at some point).
>> 
>> [0] http://dev.w3.org/html5/spec/headings-and-sections.html#outlines
> 
> As best I can tell from the HTML5 outlining algorithm, it will also capture informative headings such as Table of Contents, Abstract, Status, and introductory material.  I don't see that as a serious drawback however.  

Yes it would, but I don't think that's a problem as sections that don't have tests linked to them don't get shown in all the parts of the UI that are test-related. In fact we already have extraneous sections by the truckload. If you compare the sections that the system knows about Element Traversal:

    http://w3c-test.org/framework/app/spec/elementtraversal/sections (you need to be logged in for this one)

with those it shows in any test-related output, e.g. the results:

    http://w3c-test.org/framework/app/suite/elementtraversal/results/scope=full/group-by-section=true

you can see that in fact most of the sections are not listed there.

If we had better conventions for specification markup we could actually filter more, for instance ReSpec-generated specs all have a class of "introductory" for the front matter, and of "informative" for sections that have no normative content. But I don't think that importing too many sections is a problem and we probably shouldn't block waiting for consensus on specification markup :)

> It would be much easier to change the underlying code than to obtain working group consensus on the appearance of the spec. Spec Appearance is a classic bike-shed [1] problem.  :D

Heh, indeed :) Besides, the less we require of groups (be it bikeshedding or not) the better.

-- 
Robin Berjon - http://berjon.com/ - @robinberjon

Received on Friday, 29 June 2012 09:56:18 UTC