Re: [fullscreen] Problems with mouse-edge scrolling and games

On Mon, Feb 24, 2014 at 1:16 AM, Thibaut Despoulain
<thibaut@artillery.com>wrote:

> I've written a test for this here:
>> http://codeflow.org/issues/software-cursor.html
>>
>> My observation from testing on linux is that I can't distinguish latency
>> for the software cursor from the OS cursor (or not by much anyway) in
>> google chrome. In firefox however there's noticable lag. Mileage may vary
>> for other platforms.
>>
>
> This is true, but sadly your test runs on an empty animation frame. If
> your main thread is doing a lot of work already (barely hitting the 60fps
> mark, as it is the case for demanding games), the lag will be much more
> perceptible as you will most likely drop a frame every now and then.
>

I regard dropping below 60fps as an application defect for a realtime
interactive application where the users input is time critical.

*Reasons things drop below 60fps*

   - You trigger GC-ing
   - Your JS main thread takes up more than 16ms
   - Your GPU processing time takes up more than 16ms

*Detecting the problem*

   - You'd detect GC-ing by measuring the rate at which
   requestAnimationFrame is fired. Gaps that appear periodically are typically
   GC related.
   - JS main thread time can be accurately measure with performance.now()
   - GPU time will be measurable with EXT_disjoint_timer_query
   http://www.khronos.org/registry/webgl/extensions/EXT_disjoint_timer_query/

*Solving the problem*

   - Incremental GCs for JS will help. But they're not here yet, so one way
   you can do something right now is to be very careful about the use of [],
   {}, function(){}, new, DOM, createElement, innerHTML etc. It is possible to
   eliminate GC caused frame drop almost entirely that way.
   - Set yourself a time budget for the JS-main thread (that's well below
   16ms) that you'll want to reach even on the lower-end of the expected
   hardware. First try to keep everything below that budget in tests. If
   something simply cannot be done in that time, split up into multi-frame
   processing or shuffle it out to a webworker. Finally, if you still run into
   the problem, keep constant tap on the JS time per frame, and when you run
   over it reduce some processing (as in LOD, quality degradation etc.)
   - EXT_disjoint_timer_query is not yet available, but once it does become
   available you can use it to perform accurate testing on the lower end of
   the hardware spectrum to identify rendering issues. And you can also use it
   to dynamically react to pending performance issues by dialing down the LOD.

*Critical features for this use-case vendors have to offer, and soon*

   - Showing the OS cursor, but containing it into a bounding region as
   determined by a dom element (such as body, other elements)
   - Implementing incremental GCs
   - Implementing EXT_disjoint_timer_query
   - Reducing input -> output lag

I will point out that I have consistently called for fixes in these areas
during the last 2-3 years on the various discussions/specs/tickets on these
topics. So it's not surprising to me that these are issues now somebody
actually tries to make a real world, real time application in a browser.
I'm very glad that Artillery is doing the fine work of pushing this
boundary. I'd also like to ask the vendors to go the last mile as quickly
as feasible. Solving these issues is in the end, not just crucial for
realtime applications. Web applications get a lot of criticism for not
being able to compete in quality with native applications. This stems in
large part, from the fact that it is hard to make web applications as
responsive and snappy as native applications.