Re: test coverage data for XHR spec

Here you go, Anne:
https://github.com/whatwg/xhr/pull/17
(And I'd guesstimate perhaps half of the meta data is wrong for the WHATWG version of XHR by now - if you want, I can add the  text "(beta)" or something like that to the checkbox label).

Art:

> Wondering aloud ... adding this functionality to some of what I'll 
> characterize as "more foundational / horizontal" specs (f.ex. DOM, Web 
> IDL, ...) would be nice. Any takers out there to help with such an effort?

The main part of the effort is creating and maintaining the meta data - which is frankly quite a time consuming chore. Also, because most sensible way I've found so far to identify specific assertations is using XPath expressions, the meta data I've ended up writing obviously depends a lot on the markup of the spec and is pretty vulnerable to editorial changes. (It does help that I usually 'anchor' the XPath in the nearest header or other element with an ID, but it only goes so far). For this reason, you really want a pretty stable and mature spec before you build meta data. (And in hindsight, making this experiment for the XHR spec of all possible choices was probably a bad idea, given how much it has changed since ;)).

On the other hand, if we think this approach has value it's not insurmountable to build tools that help harvest the meta data. We have algorithms for creating XPath expressions to identify specific elements, this functionality is in all browser developer tools AFAIK. We could easily make a tool (extension?) that lets you review a specific test's source code, while instructing you to click on all assertations in the spec that seem relevant and computing the XPath expressions from your clicks. Actually, for the clipboard events spec meta data generation can be even more automated, but I'll get back to this if I manage to write some code to do that..

-Hallvord

Received on Sunday, 19 October 2014 12:29:12 UTC