Re: Point Events test submission/approval process on GitHub

On Tuesday, June 4, 2013 at 4:30 PM, Olli Pettay wrote:
> On 06/04/2013 05:00 PM, Tobie Langel wrote:
> > On Tuesday, June 4, 2013 at 3:47 PM, Olli Pettay wrote:
> > > Just wondering if there should be some way to notify implementors when bugs are fixed in the approved tests.
> > > (Came to my mind after few EventSource bugs were filed on Gecko because it doesn't pass EventSource testsuite, but some of the bugs are
> > > actually testsuite issues.)
> > 
> > If the bugs are filed on GitHub (which is what I'd recommend doing) by the implementors themselves, they'll be automatically updated when the issue is closed.
> > 
> > --tobie
> Well, the more interesting part is how to notify other implementors. Implementation A may be passing all the tests, but when B is being tested,
> it is noticed that some test if buggy and the test is changed. Now A doesn't pass the testsuite anymore. How to let A's developers to know that the
> testsuite has changed so that they can retest their implementation?
> Would be nice to have some automatic thing for this.

Got it. Some of the things we're planning like automatically running the tests using saucelabs might address that. 

That said, vendors will see they no longer pass a test the next time they pull the repository in.

--tobie

Received on Tuesday, 4 June 2013 14:48:42 UTC