Re: Proposal: moving tests to GitHub

On 1/22/13 12:37 PM, "Anne van Kesteren" <annevk@annevk.nl> wrote:

>On Tue, Jan 22, 2013 at 12:30 PM, Tobie Langel <tobie@fb.com> wrote:
>> That's definitely something to keep in mind. How frequent is it that a
>> feature moves from one spec to another (that, is outside of the
>>continuous
>> flow of features that migrate from HTML5 to WebApps)?
>>
>> Is your concern about history loss?
>
>My concern is 1) make work and 2) overhead of deciding what has to be
>tested where.

Figuring out precisely what it is you are trying to test is the crux of
the work. Having a repository per specification lightens the cognitive
load, not burdens it more. Especially for external contributors which
might not be aware of which WG handles what specs.

>E.g. tests for the ProgressEvent interface test quite a
>bit of IDL related requirements, where do they go?

Well, in that case[1], it seems the decision to put it under the
ProgressEvent directory has already been made. I don't see how this folder
being the root of a git repository or a subfolder is relevant.

With regards to the specifics of WebIDL requirements testing, there's work
going on to parse WebIDL and generate assertions out of it. Where that
isn't sufficient it seems a set of dedicated assertions (added to
testharness.js) would greatly simplify testing and answering the question
as to where these should go.

>From a distance it
>might all appear modular, but it really is all quite interconnected
>and by creating artificial boundaries that interconnectedness might
>not get testing.

These boundaries are created at the spec level. I see no harm in
replicating them at the test level. And there's certainly nothing
artificial in doing so.

That said, I agree integration testing is key, but it should be done
explicitly and methodically, not by relying on accidental coverage that's
hard to measure. The fun part about integration testing is it actually
tests the specs more than it tests the implementations themselves.

--tobie
 
---
[1]: http://w3c-test.org/webapps/ProgressEvents/tests/submissions/

Received on Tuesday, 22 January 2013 12:15:27 UTC