Re: Ref-test image flakiness when web fonts are used

Apologies that I was late to reply to the thread I asked...

​Thank you Geoffrey for sharing the background, and I'm surprised I'm
sometimes there ;-O

On Thu, Mar 9, 2017 at 1:17 AM, Geoffrey Sneddon <me@gsnedders.com> wrote:

>> I think what we need to define for web-platform-tests is that for
>
>> reftests the screenshot should be taken after the last of the
> >> following:
> >>
> >>  * after the document's load event has been fired
> >>  * the document.fonts.ready promise being fulfilled
> >>  * all @imports from stylesheets have been loaded and style recomputed
> >> etc.
> >>  * after all <uri> subresources in all stylesheets have been loaded
> >> etc. (so background-image, content, etc.)
> >>
> >> I think that's it?
>

​Yes, I think the list is correct. The font loading spec says the
document.fonts.ready promise implies layout completes, if my reading is
correct.

> The user agent has actually painted (there seems to be a bug in blink
> where
> > we sometimes take a screenshot before images have painted, even though
> load
> > has fired).
>

​Yeah, obviously we need to wait for paint, but I think it's even less
defined and obvious?


> > Are there actually web apis to determine all these things?
>
> No. Which yes, makes this awkward.
>

​So...is this something we can define or recommend somewhere? It won't be a
great situation where one vendor assumes that tests include "
/common/reftest-wait-for-fonts.js", while another vendor assumes the test
runner waits for these condition, no?