- From: Hallvord R. M. Steen <hsteen@mozilla.com>
- Date: Mon, 6 Oct 2014 04:51:26 -0700 (PDT)
- To: public-webapps <public-webapps@w3.org>
- Cc: Jungkee Song <jungkees@gmail.com>, Julian Aubourg <j@ubourg.net>
Hi,
partly quoting myself from
https://github.com/w3c/web-platform-tests/pull/1272 :
Nearly all tests in the XHR test suite have (potentially outdated) meta data linking them to specific assertations in the spec. (Technically this is a set of space-separated XPath expressions for each link @rel=help element which, combined with the section linked to from the HREF, identifies a given assertation).
I've written some new code to actually make use of this data (I used to have some, but it was an Opera UserJS script using the localStorage available to such scripts - a bit too browser-specific, and I lost it when leaving Opera anyway).
In this pull request, I add one Python script to extract the meta data into a single JSON file, and one javascript to iterate over the resulting JSON data and annotate the spec.
To test the outcome, load http://www.w3.org/TR/XMLHttpRequest/ and run this from the browser's dev tools' console:
document.body.appendChild(document.createElement('script')).src='http://hallvord.com/temp/xhr/annotate_spec.js'
You may have to disable mixed content blocking in your browser if the spec loads over https.
To be clear: the expected effect of running the script is
1) Every assertation that have at least one associated test, get a light green background
2) One or more link(s) to relevant test(s) are added after each assertation.
As I said, some of the meta data is subtly outdated - this is especially evident in the "open method" section, where most links are off by one LI (clearly, we spec authors added a LI since the meta data was edited). I will find some time to review and fix such issues. Also, sections that are in fact being tested may lack annotations.
In any case, hopefully the net outcome of this experiment is that we can state with a lot of confidence that we have a high test coverage for our spec, and that we can push coverage further by identifying corners that remain un-tested.
Please test and comment :)
-Hallvord
Received on Monday, 6 October 2014 11:51:53 UTC