Re: Proposal: adopt a "test required" policy for spec changes

Hrm. We rushed the call last week (my fault!), and ended up skipping over
this entirely.

I'm still in favor of adopting a policy along the lines of Web
Performance's, I'm still a little bit concerned about velocity, but I'm
still assuaged that an incubation-first strategy should reduce the costs as
most of the work we're doing in this WG should be polishing as opposed to
making radical changes in the direction of an unproven design.

Wendy, would adopting such a policy require any changes to formal W3C
docs/charter/etc, or would adding a CONTRIBUTING file be something we could
decide to do without asking permission? :)

-mike

On Wed, Sep 27, 2017 at 1:49 PM, Philip Jägenstedt <foolip@google.com>
wrote:

> Quite excited about this, is this blocked on having a meeting?
>
> On Mon, Sep 11, 2017 at 1:50 PM Mike West <mkwst@google.com> wrote:
>
>> I'm in favor of adopting a policy along the lines of Web Performance's.
>> Requiring tests for changes to a document in CR, in particular, seems like
>> a no-brainer, since we ought to have enough implementation experience by
>> that point to make writing tests against an implementation trivial.
>>
>> I'm a little more skeptical about a test-first approach to designing a
>> feature in the first place, though, as I see some marginal risk of locking
>> ourselves into a bad design just due to inertia and sunk cost. Chrome's
>> incubation-first strategy seems like a reasonable way of mitigating this
>> from a working group perspective (e.g. we'd only adopt documents (and
>> therefore apply a test-first policy to them)) once we were reasonably
>> confident that their general shape was itself reasonable.
>>
>> +Philip who's been thinking about this a lot, both from Chrome's
>> perspective, and from the perspective of the WHATWG (where such a policy is already
>> solidly in place <https://whatwg.org/working-mode>)).
>>
>> -mike
>>
>> On Fri, Sep 8, 2017 at 9:47 PM, Daniel Veditz <dveditz@mozilla.com>
>> wrote:
>>
>>> The W3 leadership has been emphasizing the importance of ensuring
>>> interoperability through Web Platform Tests. Some groups have adopted a
>>> policy of requiring corresponding web-platform-tests pull requests for
>>> before landing normative spec changes. Since interoperability is part of
>>> getting a spec to become a Recommendation this makes sense especially for
>>> specs that are in or entering CR. Should we adopt such a policy?
>>>
>>> The Web Performance group adopted the following:
>>> [[
>>> ALL normative spec changes are generally expected to have a
>>> corresponding pull request in web-platforms-tests, either in the form of
>>> new tests or modifications to existing tests, or must include the
>>> rationale for why test updates are not required for the proposed update.
>>> [...]
>>> ]]
>>> https://github.com/w3c/web-per <http://goog_1391446905>
>>> formance/blob/gh-pages/CONTRIB <http://goog_1391446905>UTING.md
>>>
>>> ... and the CSS Working Group adopted one last week:
>>> [[
>>> For normative changes for any specification in CR or later as well as
>>> the pre-CR specifications listed below, a corresponding web-platform-tests
>>> PR must be provided, except if testing is not practical; for other
>>> specifications it is usually appreciated. Typically, both PRs will be
>>> merged at the same time. Note that a test change that contradicts the spec
>>> should not be merged before the corresponding spec change.
>>> [...]
>>> ]]
>>> https://github.com/w3c/csswg-drafts/blob/master/CONTRIBUTING.md
>>>
>>> Good idea? Objections? Respond on list and we can talk about it on our
>>> next call (Sept 20).
>>>
>>> -Dan Veditz
>>>
>>
>>