Re: Testsuite flags

On 08/17/2016 09:48 PM, Geoffrey Sneddon wrote:
>
> I think we should drop combo. We added it for the sake of CR-exit
> criteria and because we had some tests that were just the sum of others.
> We probably can use case-by-case judgement better than we can the flag
> to do anything here.

I think this is fair.

> I think we can drop dom, because script can be statically detected
> easily enough (i.e., is there a <script> element?). I think Florian had
> some edge-case he was concerned with around here? (I want to say that
> was as far back as TPAC last year he mentioned that to me!)

The case was, IIRC, a test which is easier to perform without script
but has some scripting as an aid.

> I'd like to drop font, and vastly reduce the number of tests that
> require fonts to be installed by just relying on @font-face more (I hope
> we've reached a point where we can rely on it now!). Obviously we can't
> completely eliminate having to install fonts, but we can make it rare.

I think we can, indeed, do that. If a UA doesn't support @font-face,
then it can just assume that any test that includes @font-face and
isn't testing @font-face requires special font installation.
We should document that maybe. :)

> I'm also going to point out that as we try and converge on
> web-platform-tests policies we're going to end up requiring "-manual"
> filename suffixes on all tests flagged with animated, font, history,
> interact, paged, speech, or userstyle. We may want to drop some of these
> prior to that, however (or rather, more likely, say "these are
> deprecated and should be treated identically to interact").

I think it's reasonably likely that some UA will have an automated way
to handle animated tests at some point. So it seems reasonable to keep
that separate.

Paged tests can be automated in Gecko. The are not manual.

Speech would be good to distinguish. They're likely to be handled
in a separate run from everything else.

Userstyle, history, and interact can maybe be collapsed. Though IIRC
the first two can be automated, just not without proprietary testing
infrastructure--and for that reason we may want them kept distinct.

~fantasai

Received on Thursday, 18 August 2016 06:15:53 UTC