W3C home > Mailing lists > Public > public-test-infra@w3.org > January to March 2017

Re: Ref-test image flakiness when web fonts are used

From: Philip Jägenstedt <foolip@chromium.org>
Date: Tue, 07 Mar 2017 09:28:51 +0000
Message-ID: <CAARdPYc=mB_+UKvFFjF9hH6NDW95wMtKysCju9YLNUBQxRMhOw@mail.gmail.com>
To: Koji Ishii <kojii@chromium.org>, public-test-infra@w3.org
Cc: Geoffrey Sneddon <me@gsnedders.com>, James Graham <james@hoppipolla.co.uk>, Dominik Röttsches <drott@chromium.org>, Kunihiko Sakamoto <ksakamoto@chromium.org>
Or, perhaps better, we might have a /common/reftest-wait-for-fonts.js that
does all of that, so that no further scripts are needed in the page?

On Tue, Mar 7, 2017 at 6:16 PM Philip Jägenstedt <foolip@chromium.org>

> Hi all,
> This issue is a bit complicated, but Koji wrote a doc that might help
> understand the background:
> https://docs.google.com/document/d/11v0QEy8rvoNlFadAc4o3aaz9DMSnmRYmL4qpRJeD7Dc/edit?usp=sharing
> This boils down to what timing guarantees are made for the
> document.fonts.ready promise being resolved. AFAIU, nothing about font
> loading influences the window load event, because web font loads are
> triggered by layout.
> The specs says something interesting about the promise timing, but some
> testing revealed that it doesn't actually match implementations:
> https://github.com/w3c/csswg-drafts/issues/1082
> So, given this, I think tests would have to wait for first the window load
> event and then the document.fonts.ready promise, but that should suffice.
> Something like:
> <!DOCTYPE html>
> <html class="reftest-wait">
> <script src=/common/reftest-wait.js></script>
> <script>
> window.onload = () => {
>   document.fonts.ready.then(takeScreenshot);
> };
> </script>
> <p>something with fonts
> (not tested)
> In fact, perhaps we can just add a takeScreenshotWhenFontsReady() to
> reftest-wait.js? And we could put a TODO there to simplify depending on the
> outcome of CSS Font Loading spec discussion.
> On Sat, Mar 4, 2017 at 12:48 AM Koji Ishii <kojii@chromium.org> wrote:
> Hi all, I'm new to this ML, apologies in advance if I'm not doing good,
> appreciate to point them out.
> I've been long troubled and wondering how to solve the subject, assuming
> this is our specific problem. I happened to talk to foolip about this, who
> suggested to talk to Geoffrey and James, who suggested to move the
> discussion here. So...allow me to start by forward the first a couple of
> e-mails with their approvals.
> Hi James, Geoffrey,
> We were talking about this for a while, and when I talked to foolip, he
> suggested to reach to you two experts. Could you mind to help us?
> The problem is, when we run ref-tests that use web fonts, our bots often
> report flakiness of images. Here's an example screenshots
> <https://storage.googleapis.com/chromium-layout-test-archives/WebKit_Linux_Trusty/23985/layout-test-results/fast/text/line-break-8bit-after-16bit-diffs.html>
>  of a test
> <https://cs.chromium.org/chromium/src/third_party/WebKit/LayoutTests/fast/text/line-break-8bit-after-16bit.html?q=8bit-after&sq=package:chromium&dr>.
> Probably, on heavy load or under some conditions, web fonts are not loaded
> yet, but test runner thinks layout is finished and takes screenshots.
> We haven't reached a conclusion how it happens and how to fix it, but
> adding a script that waits for document.fonts.ready promise to resolve
> after window.onload seems to work.
> Is this something you have seen before? Could you mind to share your
> thoughts?
Received on Tuesday, 7 March 2017 09:29:36 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 17:34:13 UTC