Re: testharness.js v. reftests for rendering tests

I think that's not quite true: I think it's frequently possible to
write a smaller JS test when I wouldn't advise it, largely because of
the elimination of boilerplate (e.g., even a plain green square in the
top left of the document is a ten-line reference). Hence why I think
we want some stated preference for rendering tests, but a fairly weak
one.

/g

On Thu, Jan 5, 2017 at 3:51 PM, Philip Jägenstedt <foolip@google.com> wrote:
> I guess that amounts to "use the type of test that results in the smallest
> and most maintainable test"?
>
> On Tue, Jan 3, 2017 at 4:08 PM Geoffrey Sneddon <me@gsnedders.com> wrote:
>>
>> I wonder if the right middle-ground is some very generic comment about
>> test types: "choose between [i.e., reftests or testharness.js tests]
>> preferring the former for tests about layout and the latter for
>> everything else, but if it would be overly cumbersome to do so use the
>> other"?
>>
>> /g
>>
>> On Tue, Dec 20, 2016 at 12:51 PM, Philip Jägenstedt <foolip@google.com>
>> wrote:
>> > Right, I have used CSSOM to test the Fullscreen UA stylesheet, that was
>> > useful I think. Some reftests are still good to ensure, in the case of
>> > Fullscreen, that the top layer stuff really is on top of everything.
>> >
>> > Maybe we should advise at least one reftest to ensure each CSS property
>> > and
>> > any interesting interaction, but that tests that aren't specifically
>> > trying
>> > to test that can use CSSOM instead?
>> >
>> > On Tue, Dec 20, 2016 at 1:19 PM Simon Pieters <simonp@opera.com> wrote:
>> >>
>> >> On Tue, 20 Dec 2016 13:03:02 +0100, Geoffrey Sneddon <me@gsnedders.com>
>> >> wrote:
>> >>
>> >> > Hi,
>> >> >
>> >> > As far as I'm aware, we have no real defined policy as to how
>> >> > rendering
>> >> > tests should be written. We essentially have two options:
>> >> > testharness.js
>> >> > using the CSSOM or reftests.
>> >> >
>> >> > I believe the current Blink policy is to use the former except when
>> >> > testing paint code, and Gecko's is to use reftests for both.
>> >> >
>> >> > On the whole, despite the performance penalty, I'd much favour
>> >> > recommending reftests for both given the intrinsic link between
>> >> > rendering and painting and the various optimisations different
>> >> > implementations do to avoid invoking the various parts with different
>> >> > mutations.
>> >> >
>> >> > Of course, if someone thinks the performance penalty is too high
>> >> > maybe
>> >> > we'll have to reconsider.
>> >> >
>> >> > /gsnedders
>> >> >
>> >>
>> >> I don't have a strong opinion about policy but I will point out that it
>> >> can sometimes be useful to test both. Equivalent CSSOM does not
>> >> necessarily mean equivalent rendering and vice versa.
>> >>
>> >> As an example, for testing the UA stylesheet, it seems most useful to
>> >> first test the CSSOM for everything. But reftests can be useful for
>> >> e.g.
>> >> testing interaction of writing modes and form controls, or margin
>> >> collapsing quirks, <ol> numbering, framesets, etc.
>> >>
>> >> --
>> >> Simon Pieters
>> >> Opera Software
>> >>
>> >

Received on Thursday, 5 January 2017 18:12:25 UTC