Re: test coverage data for XHR spec

>> Please test and comment :)

> The main problem I have is that this specification is increasingly
> out-of-date with work I've done.

The meta data is certainly outdated compared to your work. The tests themselves hopefully aren't - I mean, you've been tightening the specs to be more precise and adding functionality and all that, but in general the observed behaviour should be mostly stable - right? I hope the tests and assertations are still mostly valid (and if something disagrees with your XHR+Fetch that's probably a bug we'll fix at some point) - the only issue you have should be the links between the tests and the spec itself.

> But also simple things such as accepting a loss on trying to change
> from DOMException to TypeError are not reflected. Seems very bad to
> encourage people to write tests based on that TR/ document rather than
> https://xhr.spec.whatwg.org/

I wasn't really intending to encourage people to write tests based on TR/XMLHttpRequest - I merely used it as a demo for the simple reason that it loads over HTTP while neither the dvcs.w3 nor the WHATWG does, so people could test this out without having to disable their mixed content blocking.

If you do try this on xhr.spec.whatwg.org, you'll see that quite a lot of the meta data is still valid - it does add plenty of spec links, while warning in the console about the entries that need updating to match this version of the spec. You just need to turn off mixed content blocking if you wish to test this before I have those scripts on a HTTPS URL that allows CORS requests for the JSON file..

I'll also note that this use case requires a certain level of spec stability - it's inherently better suited for a "snapshot" - type process than a "living standard" - type process. With a "snapshot" process we can say for certain that "version 2.0." of the spec and "version 2.0" of the test suite will forever have valid links. With the living standard approach, even the LINK rel=help HREFs are easily getting outdated. For that reason the meta data might always lag a bit behind your edits ;). It might be possible to use contains(text(), 'foo') - type XPath expressions more to sort of work around this, but I'm not sure if that's a good idea in the long run really. Expressions like ol[1]/li[3] could even be updated automatically by a script when the spec changes if we deem this approach valuable enough to spend such efforts on it, whereas contains() couldn't.

I'm not sure what the best way forward towards a "2.0" W3C TR snapshot of XHR is - but I think this test meta data experiment has some value and whichever way we go I'll try to update the meta data to match the TR we end up with.

Hallvord

Received on Monday, 6 October 2014 13:04:46 UTC