W3C home > Mailing lists > Public > public-test-infra@w3.org > April to June 2017

Re: Tentative tests.

From: Philip Jägenstedt <foolip@chromium.org>
Date: Wed, 24 May 2017 12:04:24 +0000
Message-ID: <CAARdPYd8gxnz7Wxtg=euqbJ+uk8hstFDouUav_pRJ7kAb2wQPg@mail.gmail.com>
To: Mike West <mkwst@google.com>, Ojan Vafai <ojan@chromium.org>
Cc: Anne van Kesteren <annevk@annevk.nl>, Boris Zbarsky <bzbarsky@mit.edu>, Mark Dittmer <markdittmer@chromium.org>, Patrick Kettner <patket@microsoft.com>, public-test-infra@w3.org
I'll withdraw that suggestion. What I think we need is some way to make
sure that tentative tests don't stay around forever, but I think we could
simply use the Git history to see when they were most recently changed and
figure out what to do with them. So if the commit message and PR has all
the relevant information, I think there's no need to also put it in the
test file.

On Wed, May 24, 2017 at 2:02 PM Mike West <mkwst@google.com> wrote:

> On Tue, May 23, 2017 at 6:17 PM, Ojan Vafai <ojan@chromium.org> wrote:
>> On Tue, May 23, 2017, 11:40 AM Mike West <mkwst@google.com> wrote:
>>> On Tue, May 23, 2017 at 5:14 PM, Anne van Kesteren <annevk@annevk.nl>
>>> wrote:
>>>> On Tue, May 23, 2017 at 3:10 PM, Mike West <mkwst@google.com> wrote:
>>>> > 1.  Put the tests in a new file with a `-tentative.*` suffix (e.g.
>>>> > `//content-security-policy/script-src/new-test-tentative.sub.html`).
>>>> This doesn't mix with -manual. Not sure if that's a problem, but
>>>> dot-delimiting might also be okay?
>>> Sure. `.tentative.*` is fine by me.
>>>> > 2.  Put the tests in a `tentative` subdirectory of an existing suite
>>>> (e.g.
>>>> > `//content-security-policy/tentative/new-script-src-test.sub.html`).
>>>> We don't assign meaning to directories elsewhere other than informally
>>>> "resources" / "support". At least, as far as I know. I think I'd
>>>> prefer just the one convention.
>>> I like the directory simply for grouping. If there's a new feature with
>>> a lot of tests, the suffixes won't sort together as a collected bunch of
>>> tests that folks who haven't hopped on board with the feature should be
>>> ignoring.
>>> But if I'm the only one who likes that idea, I'm happy to drop it in
>>> favor of the simplicity of a single approach.
>> I like this idea as well, but am also fine with starting simple and
>> adding directory support later based of real world experience.
>> Glad to see this happen. Thanks for following up.
> `.tentative.*` going once... going twice... If there are no fundamental
> objections, then I'm going to start landing patches using this convention.
> :)
> Philip suggested elsewhere that it might be a good idea to ensure that the
> tentative test files in some way refer to the discussion that they're in
> support of. I could imagine formalizing that as a `<link rel="help"
> href="[link to GitHub issue/PR]">`. WDYT?
> -mike
Received on Wednesday, 24 May 2017 12:05:14 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 17:34:13 UTC