W3C home > Mailing lists > Public > public-test-infra@w3.org > July to September 2014

Re: wptrunner and how to handle ref tests

From: Dirk Pranke <dpranke@chromium.org>
Date: Tue, 1 Jul 2014 16:15:40 -0700
Message-ID: <CAEoffTB9=UpnxrJxv2konUtQ-abYjfF50qAGgQFVtaBsRCoeEA@mail.gmail.com>
To: James Graham <james@hoppipolla.co.uk>
Cc: public-test-infra <public-test-infra@w3.org>
On Tue, Jul 1, 2014 at 9:31 AM, James Graham <james@hoppipolla.co.uk> wrote:

> On 01/07/14 17:19, Dirk Pranke wrote:
> > I certainly don't, that's true. But, if I was being a standards purist,
> it
> > seems like defining fuzzy matching criteria would be a good idea, rather
> > than leaving it be implementation defined.
> >
> > That said, I'm not being a standards purist and I'd rather focus on
> > whatever gets people running more tests more often :).
> Sure. But there are pragmatic reasons for not depending on Imagemagick.
> The simplest is that it potentially requires everyone to use the same
> version of Imagemagick, with the same compile time switches, in order to
> get the same results. This leaves a high chance of things like OS
> upgrades on test infrastructure breaking results. We might be lucky and
> get away with it, but it is a risk.

I'm sorry, I thought you were arguing that Imagemagick was a good idea, and
now I think you're suggesting the opposite?

I actually don't have a strong leaning either way, as I don't have an
answer I really like to this problem, short of understanding where the
actual diffs come from and what can be done about them.

-- Dirk
Received on Tuesday, 1 July 2014 23:16:28 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 17:34:11 UTC