Re: testharness.js v. reftests for rendering tests

I guess that amounts to "use the type of test that results in the smallest
and most maintainable test"?

On Tue, Jan 3, 2017 at 4:08 PM Geoffrey Sneddon <me@gsnedders.com> wrote:

> I wonder if the right middle-ground is some very generic comment about
> test types: "choose between [i.e., reftests or testharness.js tests]
> preferring the former for tests about layout and the latter for
> everything else, but if it would be overly cumbersome to do so use the
> other"?
>
> /g
>
> On Tue, Dec 20, 2016 at 12:51 PM, Philip Jägenstedt <foolip@google.com>
> wrote:
> > Right, I have used CSSOM to test the Fullscreen UA stylesheet, that was
> > useful I think. Some reftests are still good to ensure, in the case of
> > Fullscreen, that the top layer stuff really is on top of everything.
> >
> > Maybe we should advise at least one reftest to ensure each CSS property
> and
> > any interesting interaction, but that tests that aren't specifically
> trying
> > to test that can use CSSOM instead?
> >
> > On Tue, Dec 20, 2016 at 1:19 PM Simon Pieters <simonp@opera.com> wrote:
> >>
> >> On Tue, 20 Dec 2016 13:03:02 +0100, Geoffrey Sneddon <me@gsnedders.com>
> >> wrote:
> >>
> >> > Hi,
> >> >
> >> > As far as I'm aware, we have no real defined policy as to how
> rendering
> >> > tests should be written. We essentially have two options:
> testharness.js
> >> > using the CSSOM or reftests.
> >> >
> >> > I believe the current Blink policy is to use the former except when
> >> > testing paint code, and Gecko's is to use reftests for both.
> >> >
> >> > On the whole, despite the performance penalty, I'd much favour
> >> > recommending reftests for both given the intrinsic link between
> >> > rendering and painting and the various optimisations different
> >> > implementations do to avoid invoking the various parts with different
> >> > mutations.
> >> >
> >> > Of course, if someone thinks the performance penalty is too high maybe
> >> > we'll have to reconsider.
> >> >
> >> > /gsnedders
> >> >
> >>
> >> I don't have a strong opinion about policy but I will point out that it
> >> can sometimes be useful to test both. Equivalent CSSOM does not
> >> necessarily mean equivalent rendering and vice versa.
> >>
> >> As an example, for testing the UA stylesheet, it seems most useful to
> >> first test the CSSOM for everything. But reftests can be useful for e.g.
> >> testing interaction of writing modes and form controls, or margin
> >> collapsing quirks, <ol> numbering, framesets, etc.
> >>
> >> --
> >> Simon Pieters
> >> Opera Software
> >>
> >
>