Re: Testing getUserMedia

Two side questions on testing ...

Could we use a repo in the W3C area on github (https://github.com/w3c) instead of mercurial ?

Could you include in the test some sort of automation tool so that the text can be run in an automated form ?


On May 31, 2012, at 4:35 AM, Dominique Hazael-Massieux wrote:

> Hi,
> 
> As getUserMedia matures and is getting deployed, the need to ensure
> interoperability across implementation increases; the only right way to
> ensure that interoperability is to have tests.
> 
> While formally we only need to show interoperability during Candidate
> Recommendation, I think it's worthwhile starting to create tests now,
> even if that means that some of these tests will have to modified to
> keep up with changes in the spec.
> 
> This message tries to serve as a general intro to how we do testing at
> W3C.
> 
> At a high level, a test case for a JavaScript API is an HTML file that
> exercises a specific aspect of the API and tries to determine if the API
> behaves as specified or not when run in the browser under test.
> 
> W3C groups working in this space use a common framework to develop test
> cases that facilitate automating the run of these test cases, as well as
> the collection of results from browsers running them. That test harness
> is described at:
> http://www.w3.org/2008/webapps/wiki/Harness
> 
> Unless there is a strong reason not to, I think we too should adopt that
> harness for the development of our test cases.
> 
> Process-wise, I think we should also follow the way of other groups:
> * have someone in the group designed as the test facilitator, that
> ensures that test cases get submitted, reviewed, approved
> 
> * test cases should be submitted either by email or better by uploading
> them to a dedicated mercurial repository; I've created
> https://dvcs.w3.org/hg/media-capture/file/tip to that end, to which
> anyone in DAP and WebRTC should have read-write access
> 
> * test cases are first put into the "submitted" directory; they'll get
> moved to "approved" once the group gets a chance to review and approve
> them
> 
> * we can also accept contribution of test cases from non group
> participants; I can explain more about the logistics of this when needed
> 
> We probably need to define how we want to review and approve test cases;
> different groups have had different approaches. But that's probably
> easier done once we have found a test facilitator for the spec :)
> 
> I've started creating test cases which I hope can also serve as useful
> starting points for other contributors; I'll give more details about
> this in a separate mail.
> 
> Dom
> 
> 
>

Received on Thursday, 31 May 2012 14:21:06 UTC