Re: Box model / render tee

On Thu, Apr 17, 2014 at 9:37 AM, Brian Kardell <bkardell@gmail.com> wrote:
> I was wondering if anyone from or who follows CSSWG who attended the talks
> about the discussion on this topic that took place at the Extensible Web
> Summit could share any of those discussions back with the group.  Due to
> competing sessions i was unable to attend this one.  The minutes are
> available @ http://oksoclap.com/p/P3aS4GtR2L - I'm wondering if there was
> anything specific/useful that came out of it in terms of something the WG is
> willing to take up?

Yes, quite a bit!  It was a really useful talk, as I finally got to
understand what was being asked for when people were talking about
"expose the box tree".

So, the talk was led by some folks from the react.js project, which is
a data-flow library built atop JS and the DOM.  It holds the canonical
state of your application in JS, and treats the DOM as a render target
- if you need to adjust the value of a form element, you don't grab
the DOM element, you just look at the value slot in the react.js
object model for it.  This has some really cool features, and I would
like to make this kind of thing easier to do.

It turns out the major problem with their model is that *some* of the
state of the DOM can't be extracted out.  If you're using an <input
type=text>, for example, the precise state of the <input> isn't
knowable.  You can know gross state - the current value, etc - but not
precisely where the cursor is, what the state of the IME is, etc., and
this just gets worse with more complicated form elements and a few
other kinds of elements.

The net result of this is that react.js has a really nasty processing
model.  Rather than holding all state in JS and regenning the DOM
fresh when necessary, they sometimes have to gen up some new DOM, then
diff it against the old DOM to see if there are any shared form
elements between the two, so they can actually lift the element from
the old DOM into the new DOM.  This allows authors to continue working
with the element without interruption.  This, obviously, ends up being
slow (tree-diffing algorithms aren't very fast, by their nature), and
as stated, kinda nasty.

This led to a discussion of ways we can avoid having this state locked
up in the DOM.  We could probably expose more state on inputs and the
like, but even if we couldn't (some stuff is very complicated to
express, and of no use other than copying it into a new element), we
could still do something like allow you to ask for an opaque object
representing an element's state at some time, and then allow you to
mash this into another element of the same type, so the user can pick
up where they left off in a new element.

Coming at the problem from the other side, the major reason react.js
uses the DOM as a render target rather than WebGL is because of the
handful of user-interaction elements built into the DOM that are
super-hard to rewrite yourself to any fidelity.  Something as "simple"
as an <input type=text> represents *years* of engineer time getting
all the arcane details of text input, internationalization, IMEs, etc
correct.  While it's easy to pound out a custom text field that mostly
gets English right, that's not really good enough.  It would be
interesting to pursue the problem from this side, and allow some way
for users of <canvas> to paint complex widgets into their scene.  If
we can make a spinning dodecahedron with text inputs on every face in
3D CSS, why not in WebGL?

There was further discussion on the matter named in the title, that of
exposing the box tree more directly to authors, and making it
editable.  There was some back and forth on the details of what this
would entail, but what it probably comes down to is more work on
formalizing the layout communication model between boxes, so that
custom things can interface with it and provide their own
min-size/preferred-size/etc.

Overall, I was very happy with the talk, and glad I was able to help
distill relevant problems and use-cases from the engineers there.

~TJ

Received on Saturday, 19 April 2014 01:20:30 UTC