W3C home > Mailing lists > Public > public-test-infra@w3.org > January to March 2017

Re: Ref-test image flakiness when web fonts are used

From: Philip Jägenstedt <foolip@chromium.org>
Date: Tue, 07 Mar 2017 09:16:23 +0000
Message-ID: <CAARdPYcVW5TKrrNQBZykNiaf1PCgoLDxCrLbrTCrL=z5i72NKQ@mail.gmail.com>
To: Koji Ishii <kojii@chromium.org>, public-test-infra@w3.org
Cc: Geoffrey Sneddon <me@gsnedders.com>, James Graham <james@hoppipolla.co.uk>, Dominik Röttsches <drott@chromium.org>, Kunihiko Sakamoto <ksakamoto@chromium.org>
Hi all,

This issue is a bit complicated, but Koji wrote a doc that might help
understand the background:

This boils down to what timing guarantees are made for the
document.fonts.ready promise being resolved. AFAIU, nothing about font
loading influences the window load event, because web font loads are
triggered by layout.

The specs says something interesting about the promise timing, but some
testing revealed that it doesn't actually match implementations:

So, given this, I think tests would have to wait for first the window load
event and then the document.fonts.ready promise, but that should suffice.
Something like:

<!DOCTYPE html>
<html class="reftest-wait">
<script src=/common/reftest-wait.js></script>
window.onload = () => {
<p>something with fonts

(not tested)

In fact, perhaps we can just add a takeScreenshotWhenFontsReady() to
reftest-wait.js? And we could put a TODO there to simplify depending on the
outcome of CSS Font Loading spec discussion.


On Sat, Mar 4, 2017 at 12:48 AM Koji Ishii <kojii@chromium.org> wrote:

> Hi all, I'm new to this ML, apologies in advance if I'm not doing good,
> appreciate to point them out.
> I've been long troubled and wondering how to solve the subject, assuming
> this is our specific problem. I happened to talk to foolip about this, who
> suggested to talk to Geoffrey and James, who suggested to move the
> discussion here. So...allow me to start by forward the first a couple of
> e-mails with their approvals.
> Hi James, Geoffrey,
> We were talking about this for a while, and when I talked to foolip, he
> suggested to reach to you two experts. Could you mind to help us?
> The problem is, when we run ref-tests that use web fonts, our bots often
> report flakiness of images. Here's an example screenshots
> <https://storage.googleapis.com/chromium-layout-test-archives/WebKit_Linux_Trusty/23985/layout-test-results/fast/text/line-break-8bit-after-16bit-diffs.html>
>  of a test
> <https://cs.chromium.org/chromium/src/third_party/WebKit/LayoutTests/fast/text/line-break-8bit-after-16bit.html?q=8bit-after&sq=package:chromium&dr>.
> Probably, on heavy load or under some conditions, web fonts are not loaded
> yet, but test runner thinks layout is finished and takes screenshots.
> We haven't reached a conclusion how it happens and how to fix it, but
> adding a script that waits for document.fonts.ready promise to resolve
> after window.onload seems to work.
> Is this something you have seen before? Could you mind to share your
> thoughts?
Received on Tuesday, 7 March 2017 09:17:09 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 17:34:13 UTC