Re: Testing getUserMedia

Le jeudi 31 mai 2012 à 08:20 -0600, Cullen Jennings a écrit :
> Two side questions on testing ...
> 
> Could we use a repo in the W3C area on github (https://github.com/w3c) instead of mercurial ?

There is actually integration between the mercurial repository and our
test framework http://w3c-test.org/framework/ so that wouldn't be
entirely trivial.

That being said, I know that other groups have a dual presence on github
and dvcs.w3.org, and it might be that in general we're better served by
having this situation as well.

I assume you're asking for github to make it easier to contribute to the
test suite?

> Could you include in the test some sort of automation tool so that the text can be run in an automated form ?

That's already the case; the reason for including testharness.js and
testharnessreport.js in each test case is precisely to make automation
possible; see for instance how Mozilla are using it:
https://bugzilla.mozilla.org/show_bug.cgi?id=647323

Dom

> 
> On May 31, 2012, at 4:35 AM, Dominique Hazael-Massieux wrote:
> 
> > Hi,
> > 
> > As getUserMedia matures and is getting deployed, the need to ensure
> > interoperability across implementation increases; the only right way to
> > ensure that interoperability is to have tests.
> > 
> > While formally we only need to show interoperability during Candidate
> > Recommendation, I think it's worthwhile starting to create tests now,
> > even if that means that some of these tests will have to modified to
> > keep up with changes in the spec.
> > 
> > This message tries to serve as a general intro to how we do testing at
> > W3C.
> > 
> > At a high level, a test case for a JavaScript API is an HTML file that
> > exercises a specific aspect of the API and tries to determine if the API
> > behaves as specified or not when run in the browser under test.
> > 
> > W3C groups working in this space use a common framework to develop test
> > cases that facilitate automating the run of these test cases, as well as
> > the collection of results from browsers running them. That test harness
> > is described at:
> > http://www.w3.org/2008/webapps/wiki/Harness
> > 
> > Unless there is a strong reason not to, I think we too should adopt that
> > harness for the development of our test cases.
> > 
> > Process-wise, I think we should also follow the way of other groups:
> > * have someone in the group designed as the test facilitator, that
> > ensures that test cases get submitted, reviewed, approved
> > 
> > * test cases should be submitted either by email or better by uploading
> > them to a dedicated mercurial repository; I've created
> > https://dvcs.w3.org/hg/media-capture/file/tip to that end, to which
> > anyone in DAP and WebRTC should have read-write access
> > 
> > * test cases are first put into the "submitted" directory; they'll get
> > moved to "approved" once the group gets a chance to review and approve
> > them
> > 
> > * we can also accept contribution of test cases from non group
> > participants; I can explain more about the logistics of this when needed
> > 
> > We probably need to define how we want to review and approve test cases;
> > different groups have had different approaches. But that's probably
> > easier done once we have found a test facilitator for the spec :)
> > 
> > I've started creating test cases which I hope can also serve as useful
> > starting points for other contributors; I'll give more details about
> > this in a separate mail.
> > 
> > Dom
> > 
> > 
> > 
> 
>

Received on Friday, 1 June 2012 09:39:35 UTC