Re: testharness.js v. reftests for rendering tests

Right, I have used CSSOM to test the Fullscreen UA stylesheet, that was
useful I think. Some reftests are still good to ensure, in the case of
Fullscreen, that the top layer stuff really is on top of everything.

Maybe we should advise at least one reftest to ensure each CSS property and
any interesting interaction, but that tests that aren't specifically trying
to test that can use CSSOM instead?

On Tue, Dec 20, 2016 at 1:19 PM Simon Pieters <simonp@opera.com> wrote:

> On Tue, 20 Dec 2016 13:03:02 +0100, Geoffrey Sneddon <me@gsnedders.com>
> wrote:
>
> > Hi,
> >
> > As far as I'm aware, we have no real defined policy as to how rendering
> > tests should be written. We essentially have two options: testharness.js
> > using the CSSOM or reftests.
> >
> > I believe the current Blink policy is to use the former except when
> > testing paint code, and Gecko's is to use reftests for both.
> >
> > On the whole, despite the performance penalty, I'd much favour
> > recommending reftests for both given the intrinsic link between
> > rendering and painting and the various optimisations different
> > implementations do to avoid invoking the various parts with different
> > mutations.
> >
> > Of course, if someone thinks the performance penalty is too high maybe
> > we'll have to reconsider.
> >
> > /gsnedders
> >
>
> I don't have a strong opinion about policy but I will point out that it
> can sometimes be useful to test both. Equivalent CSSOM does not
> necessarily mean equivalent rendering and vice versa.
>
> As an example, for testing the UA stylesheet, it seems most useful to
> first test the CSSOM for everything. But reftests can be useful for e.g.
> testing interaction of writing modes and form controls, or margin
> collapsing quirks, <ol> numbering, framesets, etc.
>
> --
> Simon Pieters
> Opera Software
>
>