Re: Defacto tests (Was: Tentative tests)

On Wed, May 24, 2017 at 7:12 PM, Philip Jägenstedt <foolip@chromium.org>
wrote:

> On Wed, May 24, 2017 at 7:05 PM Anne van Kesteren <annevk@annevk.nl>
> wrote:
>
>> On Wed, May 24, 2017 at 5:54 PM, Rick Byers <rbyers@chromium.org> wrote:
>> > In terms of the semantics of directories, haven't we said previously
>> that a
>> > "defacto" sub-directory (or filename suffix) is OK for tests which pass
>> in
>> > practice but aren't properly specified yet?
>>
>> I haven't seen that in the documentation.
>>
>>
>> My suggestion to Mike West is put tentative in the documentation (and
>> maybe some of the test runner stuff needs updating to ignore these
>> resources although I guess that thing was broken anyway). That makes
>> it more final and provides a clear path for any appeals.
>>
>
> It looks like "defacto" hasn't been discussed on this list.
>

Yeah perhaps it was only at the last TPAC meeting (notes here
<https://docs.google.com/document/d/1ehJAwL03NM-ptSbw7rDUo_GkiaZBrxanMrvjjqtuqis/edit#heading=h.u47f73rabp5b>
).


> A top-level defacto/ directory would be a bit worrisome if nobody wanted
> to maintain it and try to explain with specs over time. Inside
> well-maintained test suites for well-maintained specs I think it'd probably
> be fine, though, functioning like a TODO and a regression test at once.
>
> I've not been involved in any of the previous discussion, are there a few
> concrete examples by which to judge this idea?
>

My favorite example is hit-testing.  hit-testing is largely interoperable
already, and it's usually fairly obvious what the correct behavior is, but
it would likely be a huge effort to spec properly.  However there are some
special cases, and engines do occasionally make changes to align between
browsers.  In those cases it totally seems worth the effort to capture some
of the discussion and web compat lessons in tests, even if we can't justify
the cost of writing a full hit-testing spec.