Re: Testsuite flags

> On Aug 18, 2016, at 06:48, Geoffrey Sneddon <me@gsnedders.com> wrote:
> 
> I believe we have rough consensus to drop:
> 
> * ahem
> * http
> * image
> * namespace
> * svg
> * 32bit
> * 96dpi
> 

Agreed.

> Now, without rough consensus:
> 
> I think we should drop combo. We added it for the sake of CR-exit
> criteria and because we had some tests that were just the sum of others.
> We probably can use case-by-case judgement better than we can the flag
> to do anything here.

Agreed.

> I think we can drop dom, because script can be statically detected
> easily enough (i.e., is there a <script> element?). I think Florian had
> some edge-case he was concerned with around here? (I want to say that
> was as far back as TPAC last year he mentioned that to me!)

More of a curiosity than a concern. It is possible to use scripts in a test to make it easier to run (automate part of it to reduce the number of manual steps) while still keeping the test valid and usable if there is no support for script. This is a valid case for using scripts but not applying the dom flag. However, that's really a corner case of limited significance, so I'm OK with dropping the dom flag and inferring it from the presence of script. The false positives will be few and far between, and of little consequence.

> I'd like to drop font, and vastly reduce the number of tests that
> require fonts to be installed by just relying on @font-face more (I hope
> we've reached a point where we can rely on it now!). Obviously we can't
> completely eliminate having to install fonts, but we can make it rare.
> 
> I'm also going to point out that as we try and converge on
> web-platform-tests policies we're going to end up requiring "-manual"
> filename suffixes on all tests flagged with animated, font, history,
> interact, paged, speech, or userstyle. We may want to drop some of these
> prior to that, however (or rather, more likely, say "these are
> deprecated and should be treated identically to interact").

I am not sure that speech and paged fall into the same category as the rest.

I agree with merging all the other types of "can't automate fully" (animated, font, history, interact, userstyle) flags into a single category. We should continue to recognize these flags going forward for legacy reasons, but we can treat them as equivalent. Going forward, we can use the "interact" flag, or go with using a -manual suffix in the file name, or allow either. I don't care strongly.

 - Florian

Received on Tuesday, 23 August 2016 07:33:34 UTC