W3C home > Mailing lists > Public > public-test-infra@w3.org > October to December 2017

Re: Tentative tests.

From: Philip Jägenstedt <foolip@chromium.org>
Date: Tue, 10 Oct 2017 22:16:53 +0000
Message-ID: <CAARdPYfGRkkfc7Di5udon-BMOasoPMjp+SQ=SVgGkzHpgyQruw@mail.gmail.com>
To: Mike West <mkwst@google.com>, Ojan Vafai <ojan@chromium.org>
Cc: Anne van Kesteren <annevk@annevk.nl>, Boris Zbarsky <bzbarsky@mit.edu>, Mark Dittmer <markdittmer@chromium.org>, Patrick Kettner <patket@microsoft.com>, public-test-infra@w3.org
It's been a while and tentative tests are now actually in use. I'm trying
to document it in https://github.com/w3c/web-platform-tests/pull/7663,
review appreciated.

On Wed, May 24, 2017 at 2:04 PM Philip Jägenstedt <foolip@chromium.org>

> I'll withdraw that suggestion. What I think we need is some way to make
> sure that tentative tests don't stay around forever, but I think we could
> simply use the Git history to see when they were most recently changed and
> figure out what to do with them. So if the commit message and PR has all
> the relevant information, I think there's no need to also put it in the
> test file.
> On Wed, May 24, 2017 at 2:02 PM Mike West <mkwst@google.com> wrote:
>> On Tue, May 23, 2017 at 6:17 PM, Ojan Vafai <ojan@chromium.org> wrote:
>>> On Tue, May 23, 2017, 11:40 AM Mike West <mkwst@google.com> wrote:
>>>> On Tue, May 23, 2017 at 5:14 PM, Anne van Kesteren <annevk@annevk.nl>
>>>> wrote:
>>>>> On Tue, May 23, 2017 at 3:10 PM, Mike West <mkwst@google.com> wrote:
>>>>> > 1.  Put the tests in a new file with a `-tentative.*` suffix (e.g.
>>>>> > `//content-security-policy/script-src/new-test-tentative.sub.html`).
>>>>> This doesn't mix with -manual. Not sure if that's a problem, but
>>>>> dot-delimiting might also be okay?
>>>> Sure. `.tentative.*` is fine by me.
>>>>> > 2.  Put the tests in a `tentative` subdirectory of an existing suite
>>>>> (e.g.
>>>>> > `//content-security-policy/tentative/new-script-src-test.sub.html`).
>>>>> We don't assign meaning to directories elsewhere other than informally
>>>>> "resources" / "support". At least, as far as I know. I think I'd
>>>>> prefer just the one convention.
>>>> I like the directory simply for grouping. If there's a new feature with
>>>> a lot of tests, the suffixes won't sort together as a collected bunch of
>>>> tests that folks who haven't hopped on board with the feature should be
>>>> ignoring.
>>>> But if I'm the only one who likes that idea, I'm happy to drop it in
>>>> favor of the simplicity of a single approach.
>>> I like this idea as well, but am also fine with starting simple and
>>> adding directory support later based of real world experience.
>>> Glad to see this happen. Thanks for following up.
>> `.tentative.*` going once... going twice... If there are no fundamental
>> objections, then I'm going to start landing patches using this convention.
>> :)
>> Philip suggested elsewhere that it might be a good idea to ensure that
>> the tentative test files in some way refer to the discussion that they're
>> in support of. I could imagine formalizing that as a `<link rel="help"
>> href="[link to GitHub issue/PR]">`. WDYT?
>> -mike
Received on Tuesday, 10 October 2017 22:17:29 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 17:34:13 UTC