Re: [public-houdini] <none>

On Mon, Mar 9, 2015 at 9:02 PM Robert O'Callahan <robert@ocallahan.org>
wrote:

> tl;dr: please read the whole thing before commenting :-)
>
> A few weeks ago in Sydney there was a lot of discussion (mostly outside
> the formal meetings) about various proposals on the table for enabling rich
> graphical effects for mobile Web apps (e.g. UI Workers, animation-timeline,
> beforescroll). I'm not sure if Houdini is the right place for this
> discussion, but Dean assured me it is :-).
>
> Here's what I think is the key issue: App developers want a single thread
> running application code at "60fps", assembling each visual frame presented
> to the user. This thread needs to be able to observe pointer events, sample
> and modify animations, and communicate with the rest of the application.
> For the sake of discussion, let's call this the "control thread".
>

Thank you for starting this discussion! (And sorry for taking so long to
respond.) I agree that this is a key problem.


> How can we provide it in the Web platform? There seem to be two possible
> approaches:
> 1) Make the HTML5 task-loop thread the control thread. (Let's consider
> just a single document for now, for the sake of discussion.)
> 2) Make some other thread the control thread. This implies some kind of
> Worker to run JS on it. This doesn't exist in specs yet.
>
> If we choose approach #1, we will need to design some new main-thread APIs
> and do a lot of browser engineering work to make it work (to the extend
> that it *can* work). If we choose approach #2, we will need new APIs for
> this new kind of Worker. We could do both, though that would dilute
> spec/implementation efforts and our message to developers. We have to make
> a choice, even if the choice is "both".
>

I’m not sure there’s consensus on my team about the choice of control
thread (it's a pretty deep question), but I think we should experiment with
both approaches, perhaps behind a flag. It’s tough to predict which would
be most valuable in the long term. I think trying to build real things with
each approach and getting real developer feedback will be crucial for us to
figure this out.

>
> === Observations ===
>
> #1 seens fragile when you're composing a complex document. For example,
> it's not possible to create an animation library that will reliably hit
> performance targets independent of what the rest of the application is
> doing. #2 could be designed to support this.
>
> With #1, enabling Web developers to hit performance targets consistently
> across browsers, devices, and the full range of Web APIs seems like it will
> be very hard.
>
> Although #2 probably requires more new APIs, from my point of view it's
> probably less implementation effort on the browser side than getting #1 to
> work well.
>

It’s also maybe worth mentioning that if the new APIs are small, we reduce
the opportunity for compatibility bugs.

>
> With #2, if you think of the control thread being your primary application
> thread, all the async APIs you could want are available, by postMessaging
> to the main thread.
>
> For #2, we can provide most existing APIs to a control thread Worker ---
> except DOM manipulation (and by extension, full manipulation of CSS
> layout). For example, if we want to animate CSS 'width' at 60fps with text
> wrapping and border redrawing, we need #1. To some extent, #2 leads
> developers away from using CSS and the DOM. It seems very unappealing to
> replicate features like editable content, accessibility, text input and
> rich text layout outside of CSS and the DOM, so if we end up wanting these
> on the control thread, that's a problem for #2.
>

This is a fair concern, but I don’t think we’d have to reimplement all
these features. As I understand it, approach #2 would still permit an
application to be constructed in terms of components built out of DOM/CSS
with accessibility, input etc. And as you mentioned earlier, we’ll have
access to all of these from the control thread, though asynchronously.

>
> A critical question is whether #2 leads to a coherent application
> development approach which integrates 60fps application rendering with
> chunks of not-60fps CSS-rendered content, or whether it leads to an
> architecture which is not compelling to developers, or whether it leads to
> a bifurcation in the Web where apps and documents use separate APIs (and
> how bad would that be?).
>
> Personally I lean towards #2 currently, but I don't know enough to predict
> the implications.
>

Fantastic questions. I don’t think that #2 precludes components, even if
they disagree about the choice of control thread, provided we’re willing to
stomach some asynchrony in their interactions.

About the bifurcation -- there will certainly be an asymmetry in
programming paradigms, but that might actually be a benefit. In an
application where some code is performance-critical and some isn’t, the way
you write the fast stuff is often very different (eg, shaders). This is
handy both because it reminds you that you’re writing performance-critical
code and because the manner in which you’re forced to write this code can
make it tough to cause performance problems. It’s designed to be performant
and to minimize footguns.

As for whether it leads to an architecture which is not compelling to
developers, it’s really tough to say. The approach should permit
incremental (pay-to-play) adoption of performance isolation (so if not
compelling, hopefully not scary) and it seems like it might benefit some
existing frameworks, but I'd really like to do some experiments, build some
real, non-toy things, and see how it feels in practice.

Ian

>
> Rob
> --
> oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
> owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
> osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
> owohooo
> osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
> oioso
> oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
> owohooo
> osoaoyoso,o o‘oYooouo ofooooolo!o’o owoiololo oboeo oiono odoaonogoeoro
> ooofo
> otohoeo ofoioroeo ooofo ohoeololo.
>