Re: Requiring tests for spec changes

I am in favor of this.

> On Sep 4, 2017, at 3:31 AM, James Graham <james@hoppipolla.co.uk> wrote:
> 
> Now that L1 is mostly done, it's time to start thinking about how we can learn from the experience and improve the process for future work.
> 
> One thing that is being adopted by other specs to great success is to require that spec changes come with an accompanying test change, where possible (i.e. excluding things that don't affect the normative requirements). This has a number of advantages:
> 
> * We have confidence that the spec has at least basic tests for everything.
> * We don't get stuck in the current situation of trying to retro-fit tests onto an existing spec.
> * It is immediately clear to implementors if a spec change requires a corresponding change in their implementation.
> * It is clear to spec authors if they are introducing a change that unexpectedly breaks existing implementations.
> 
> I think the last points in particular are compelling. Although this seems like more work upfront, the feedback I have heard from specs like HTML and DOM is that requiring tests along with spec changes speeds up the overall specification process, because it's more likely that changes stick and aren't reverted later due to unforeseen compat issues. It also means that specification changes are more likely to reflected in implementations quickly (since there's an obvious failing test to fix), so it helps avoid the situation where a change is made and then implementations do nothing (because they simply didn't notice) and eventually it is reverted (or, worse, some implementations notice and make a fix and others don't, and clients end up relying on the behavioural differences).
> 
> I strongly recommend that we adopt this requirement or writing tests for all post-L1 spec changes.
>

Received on Tuesday, 5 September 2017 19:28:05 UTC