Re: WPT test dependencies

Den 31. jan. 2018 09:27, skrev Sergio Garcia Murillo:
> When I first read the mail, I was quite in agreement with Philipp, but
> the more I think about it, the less I am.. :)
> 
> Regarding reducing dependencies with other parts of the spec, i think it
> is a good goal, and we should do it if the alternative provides the same
> validity of the result. If the alternative is parsing the SDP or not
> testing it, then I would strongly oppose to that.
> 
> I agree that most of the test in WPT test the js api layer, like
> validating that if I set a value, then I get the same value back, and
> not the effect of that setting, but in order to test that part we would
> need some kind of external server that test IETF stuff compliance. I
> found that piece missing and really important (for testing simulcast for
> example), although I am not sure if that testing should be in the scope
> of the WPT or on something different (KITE?).


It is certainly in-scope for KITE. (That's my generic answer when the
question involves interoperability or Net-facing behavior).

> 
> Regarding changing new RTCPeerConnection() by new
> RTCPeerConnection(null) just because some browser does "a silly thing"
> and is not spec compliant, well.. We should not hack the test so a
> certain browser can pass it, but viceversa.
> 
> One last comment, I think we should be more careful when making changes
> to the spec that potentially breaks WPTs and if not providing a change
> to the WPT itself, at least open an issue so we can track those changes.

Our theoretical policy (after the last TPAC) is that a spec change
request needs to have a test change request along with it.
So far, we've not been very good about policing that.

Received on Wednesday, 31 January 2018 13:48:23 UTC