Re: Proposal: adopt a "test required" policy for spec changes

Quite excited about this, is this blocked on having a meeting?

On Mon, Sep 11, 2017 at 1:50 PM Mike West <mkwst@google.com> wrote:

> I'm in favor of adopting a policy along the lines of Web Performance's.
> Requiring tests for changes to a document in CR, in particular, seems like
> a no-brainer, since we ought to have enough implementation experience by
> that point to make writing tests against an implementation trivial.
>
> I'm a little more skeptical about a test-first approach to designing a
> feature in the first place, though, as I see some marginal risk of locking
> ourselves into a bad design just due to inertia and sunk cost. Chrome's
> incubation-first strategy seems like a reasonable way of mitigating this
> from a working group perspective (e.g. we'd only adopt documents (and
> therefore apply a test-first policy to them)) once we were reasonably
> confident that their general shape was itself reasonable.
>
> +Philip who's been thinking about this a lot, both from Chrome's
> perspective, and from the perspective of the WHATWG (where such a policy is already
> solidly in place <https://whatwg.org/working-mode>)).
>
> -mike
>
> On Fri, Sep 8, 2017 at 9:47 PM, Daniel Veditz <dveditz@mozilla.com> wrote:
>
>> The W3 leadership has been emphasizing the importance of ensuring
>> interoperability through Web Platform Tests. Some groups have adopted a
>> policy of requiring corresponding web-platform-tests pull requests for
>> before landing normative spec changes. Since interoperability is part of
>> getting a spec to become a Recommendation this makes sense especially for
>> specs that are in or entering CR. Should we adopt such a policy?
>>
>> The Web Performance group adopted the following:
>> [[
>> ALL normative spec changes are generally expected to have a
>> corresponding pull request in web-platforms-tests, either in the form of
>> new tests or modifications to existing tests, or must include the
>> rationale for why test updates are not required for the proposed update.
>> [...]
>> ]]
>> https://github.com/w3c/web-per <http://goog_1391446905>
>> formance/blob/gh-pages/CONTRIB <http://goog_1391446905>UTING.md
>>
>> ... and the CSS Working Group adopted one last week:
>> [[
>> For normative changes for any specification in CR or later as well as the
>> pre-CR specifications listed below, a corresponding web-platform-tests PR
>> must be provided, except if testing is not practical; for other
>> specifications it is usually appreciated. Typically, both PRs will be
>> merged at the same time. Note that a test change that contradicts the spec
>> should not be merged before the corresponding spec change.
>> [...]
>> ]]
>> https://github.com/w3c/csswg-drafts/blob/master/CONTRIBUTING.md
>>
>> Good idea? Objections? Respond on list and we can talk about it on our
>> next call (Sept 20).
>>
>> -Dan Veditz
>>
>
>