Re: Minimal event timing proposal

I think it's fine to continue down this path, as long as the 50ms threshold
is configurable.

Just to clarify:
-- There's a computer-side display latency (e.g. various driver-based
synchronization APIs, often very co-operative with the display, such as
2-way communications with GSYNC signaling) and a display-side display
latency (which cannot be detected).

There are situations where adjusting 50ms down to sub-1ms threshold is
favorable in specific special applications (or via a permissions API).
Events might need to be emitted in real time at sub-millisecond levels
(e.g. 2000 events per second from a high-frequency input device emitted at
0.5ms intervals etc).  Obviously, this isn't necessary for the vast
majority of applications but the standardization path should be
protected-for for the configurability of this threshold to abnormally small
values on a case-by-case basis where decoupling the input event rate from
the display refresh rate, and allowing real-time non-coalesced
notifications, is beneficial for certain kinds of high-performance
applications (Informing VR renderers earlier of input, rather than
buffering-and-coalescing, helps it prepare certain types of renderers in
advance, even if frame rate is much lower than input rate -- e.g. it's also
useful in certain low-lag programming techniques to have realtime stream of
single-events at 2000 separate events per second of a 2000 Hz input device
-- even if rendering is only, say, 90 frames per second).




On Tue, Mar 27, 2018 at 1:09 PM, Timothy Dresser <tdresser@chromium.org>
wrote:

> Long term, I think enabling reducing the threshold (for event timing and
> long tasks) makes sense, but I don't think we should block initial
> standardization on this.
>
> This proposal considers measuring the duration between event dispatch
> until the display actually updates to be out of scope.
> I believe we should land what I've got proposed, and then consider adding
> an additional "nextPaint" (name TBD) attribute as a followup.
>
> This API provides significant value without knowledge of the display time,
> and including the display time introduces a bunch of additional complexity.
>
> The primary counter-argument is that it would make more sense to threshold
> on the time from the event's hardware timestamp until the display is
> updated, instead of just until event processing is finished. If an event is
> handled quickly, but it causes a slow frame to be produced, we won't report
> a record.
>
> There are a few ways we can deal with this:
>
>    - We could change to threshold on the duration until display when we
>    spec the "nextPaint" attribute.
>    - We could rely on the slow frames frames API, which we'll eventually
>    need to spec to handle slow animations that aren't driven by input.
>
> If others think we should require exposing the time until the display is
> updated before shipping this API, I'm happy to add it, but I anticipate it
> slowing progress significantly and would rather ship a targeted initial API
> and then iterate.
>
> On Mon, Mar 26, 2018 at 5:04 PM Todd Reifsteck <toddreif@microsoft.com>
> wrote:
>
>> Mark,
>>
>> I’m not sure exactly how that feedback relates to Tim’s proposal. Can you
>> give specific and direct feedback on Tim’s proposal if I’m not accurately
>>
>>
>>
>> My translation of the feedback being given on this specification:
>>
>>    - Could a site enable this at < 50 ms when using this for high speed
>>    animations/games on well-tuned sites?
>>       - Yes, but it may trigger a lot more often than many web sites
>>       want to measure. Perhaps this could default to 50 ms, but allow a site to
>>       opt in to < 50 ms for scenarios such as what Mark is describing?
>>    - How do we measure the duration between the event updating the DOM
>>    and the display actually showing it?
>>       - Tim?
>>
>>
>>
>> With regard to the general statements made:
>>
>> Having the point data at much higher precision than 1/frame should allow
>> all movement data to be used when processing. If the screen is only updated
>> at 60 Hz, any updates to the actual graphics occurring more often than 60
>> Hz can often cause “double resource usage” when animations are thrown away
>> multiple times a frame. The purpose of allowing input callbacks to receive
>> all input data 1/paint loop is to allow the “best of both worlds” by
>> allowing the site to ensure the single paint uses all input data when
>> calculating the UI updates. If this is not true in real world usage, I
>> think the input and rendering teams would be interested to hear exactly how
>> BUT that is not the focus of this specification. (Lets please start a new
>> thread if that is a topic we’d like to discuss.)
>>
>>
>>
>> Hope that helps!
>>
>> Todd
>>
>>
>>
>> *From:* blurbusters@gmail.com <blurbusters@gmail.com> * On Behalf Of *Mark
>> Rejhon
>> *Sent:* Monday, March 26, 2018 1:47 PM
>> *To:* Timothy Dresser <tdresser@chromium.org>
>> *Cc:* Ilya Grigorik <igrigorik@google.com>; public-web-perf@w3.org
>> *Subject:* Re: Minimal event timing proposal
>>
>>
>>
>> Regarding First Input Delay, we do lots of input latency measurements as
>> Blur Busters, albiet from other points of views (e.g. gaming latency).
>> Even 5ms makes a big difference in certain contexts.  Over the long term,
>> 1000Hz mice should be streamed directly in an atomic manner to Javascript
>> -- instead of always permanently only coalesced into PointerEvents API.
>>
>>
>>
>> Microsoft Resarch has found that realtime processing of 1000Hz input make
>> a huge difference:
>>
>> https://www.youtube.com/watch?v=vOvQCPLkPt4
>>
>>
>>
>> Even on a 60Hz display it can still have large benefits.
>>
>>
>>
>> That said, understandably, while battery power can be a concern, Safari
>> has been limiting lots of processing to 60Hz even on the new 120Hz iPads,
>> certainly should be fleshed out better in all of this standardization
>> work.  Many things like requestAnimationFrame() in Safari only runs at
>> 60fps even on the 120Hz iPads, even though Section 7.1.4.2 of HTML 5.2
>> recommends it running at full frame rates on higher-Hz displays (like in
>> other browsers, Chrome, FireFox, Opera, and even some versions of Edge).
>>  These add unwanted input lag to touchscreen events on Safari.   These are
>> major input lag considerations, adding +8ms of input lag to browser apps
>> rendered in <canvas> by running rAF() at 60fps instead of 120fps on the
>> 120Hz iPads.   The display-side equation is part of the lag arithmetic too,
>> even though there are also power-consumption-versus-lag tradeoffs.
>>
>>
>>
>>
>>
>> On Mon, Mar 26, 2018 at 4:26 PM, Timothy Dresser <tdresser@chromium.org>
>> wrote:
>>
>> I've updated the proposal on WICG here
>> <https://github.com/WICG/event-timing/blob/master/README.md>.
>> This proposal requires only dispatching PerformanceEventTiming entries if
>> the duration between event startTime and when event processing is finished
>> is > 50ms.
>>
>>
>>
>> Tim
>>
>> On Fri, Feb 16, 2018 at 1:43 PM Ilya Grigorik <igrigorik@google.com>
>> wrote:
>>
>> Tim, thanks for drafting this! I like where this is headed.
>>
>>
>>
>> Left a few questions in the doc and added this to our agenda for next
>> design call
>> <https://docs.google.com/document/d/10dz_7QM5XCNsGeI63R864lF9gFqlqQD37B4q8Q46LMM/edit#heading=h.ljsbtcikd3cl> (date
>> TBD).
>>
>>
>>
>> On Thu, Feb 15, 2018 at 1:53 PM, Timothy Dresser <tdresser@chromium.org>
>> wrote:
>>
>> Based on our discussion on First Input Delay
>> <https://docs.google.com/document/d/1Tnobrn4I8ObzreIztfah_BYnDkbx3_ZfJV5gj2nrYnY/edit> at
>> the last WG meeting, I've put together a minimal proposal
>> <https://docs.google.com/document/d/10CdRCrUQzQF1sk8uHmhEPG7F_jcZ2S3l9Zm40lp3qYk/edit#heading=h.fbdd8nwxr7v4> for
>> an event timing API.
>>
>>
>>
>> The extensions to the DOM spec are fairly straight forward, and the API
>> itself is pretty bare bones. The main question is whether or not
>> dispatching an entry per DOM event is too expensive.
>>
>> If it is, we'll need to devise a method to only report a subset of events.
>>
>> I'd appreciate any feedback you have,
>> Tim
>>
>>
>>
>>
>>
>