Imperative API for Node Distribution in Shadow DOM (Revisited)

Hi all,

In today's F2F, I've got an action item to come up with a concrete workable proposal for imperative API.  I had a great chat about this afterwards with various people who attended F2F and here's a summary.  I'll continue to work with Dimitri & Erik to work out details in the coming months (our deadline is July 13th).

https://gist.github.com/rniwa/2f14588926e1a11c65d3 <https://gist.github.com/rniwa/2f14588926e1a11c65d3>

Imperative API for Node Distribution in Shadow DOM

There are two approaches to the problem depending on whether we want to natively support redistribution or not.

To recap, a redistribution of a node (N_1) happens when it's distributed to an insertion point (I_1) inside a shadow root (S_1), and I_1's parent also has a shadow root which contains an insertion point which ends picking up N_1. e.g. the original tree may look like:

(host of S_1) - S_1
  + N_1         + (host of S_2) - S_2
                   + I_1           + I_2
Here, (host of S_1) has N_1 as a child, and (host of S_2) is a child of S_1 and has I_1 as a child. S_2 has I_2 as a child. The composed tree, then, may look like:

(host of S_1)
 + (host of S_2)
   + I_2
     + N_1
 <https://gist.github.com/rniwa/2f14588926e1a11c65d3#redistribution-is-implemented-by-authors>Redistribution is implemented by authors

In this model, we can add insertAt and remove on content element and expose distributedNodes defined as follows:

insertAt(Node nodeToDistribute, long index) - Inserts nodeToDistribute to the list of the distributed nodes at index. It throws if nodeToDistribute is not a descendent (or a direct child if wanted to keep this constraint) of the shadow host of the ancestor shadow root of containt or if index is larger than the length of distributedNodes.
remove(Node distributedNode) - Remove distributedNode from the list distributed nodes. Throws if distributedNodes doesn't contain this node.
distributedNodes - Returns an array of nodes that are distributed into this insertion point in the order they appear.
In addition, content fires a synchrnous distributionchanged event when distributedNodeschanges (in response to calls to insertAt or remove). 

 <https://gist.github.com/rniwa/2f14588926e1a11c65d3#pros>Pros

Very simple / very primitive looking.
Defers the exact mechanism/algorithm of re-distributions to component authors.
We can support distributing any descendent, not just direct children, to any insertion points. This was not possible with select attribute especially with the presence of multiple generations of shadow DOM due to perfomance problems.
Allows distributed nodes to be re-ordered (select doesn't allow this).
 <https://gist.github.com/rniwa/2f14588926e1a11c65d3#cons>Cons

Each component needs to manually implement re-distributions by recursively traversing through distributedNodes of content elements inside distributedNodes of the content element if it didn't want to re-distribute everything. This is particularly challenging because you need to listen to distributionchanged event on every such content element. We might need something aking to MutationObserver's subtree option to monitor this if we're going this route.
It seems hard to support re-distribution natively in v2.
 <https://gist.github.com/rniwa/2f14588926e1a11c65d3#redistribution-is-implemented-by-uas>Redistribution is implemented by UAs

In this model, the browser is responsible for taking care of redistributions. Namely, we would like to expose distributionPool on the shadow root which contains the ordered list of nodes that could be distributed (because they're direct children of the host) or re-distributed. Conceptually, you could think of it as a depth first traversal of distributedNodes of every content element. Because this list contains every candidate for (re)distribution, it's impractical to include every descendent node especially if we wanted to do synchronous updates so we're back to supporting only direct children for distribution.

In this proposal, we add a new callback distributeCallback(NodeList distributionPool) as an arguemnt (probably inside a dictionary) to createShadowRoot. e.g.

var shadowRoot = element.createShadowRoot({
  distributedCallback: function (distributionPool) {
    ... // code to distribute nodes
  }
});
Unfortunately, we can't really use insertAt and remove in model because distributionPoolmaybe changed under the foot by (outer) insertion points in the light DOM if this shadow root to attached to a host inside another shadow DOM unless we manually listen to distributionchangedevent on every content (which may recursively appear in distributedNodes of those content).

One way to work around this problem is let UA also propagate changes to distributionPool to each nested shadow DOM. That is, when distributionPool of a shadow root gets modified due to changes to distributionPools of direct children (of the shadow host) that are content elements themselves, UA will automatically invoke distributedCallback to trigger a distribution.

We also expose distribute() on ShadowRoot to allow arbitrary execution (e.g. when its internal state changes) of this distribution propagation mechanism. Components will use this function to listen to changes in DOM.

We could also trigger this propagation mechanism at the end of micro task (via MutationObserver) when direct children of a shadow host is mutated.

In terms of actual distribution, we only need to expose add(Node) on content element. Because all candidates are distributed each time, we can clear distributed nodes from every insertion point in the shadow DOM. (Leaving them in tact doesn't make sense because some of the nodes that have been distributed in the past may no longer be available).

There is an alternative approach to add something like done() or redistribute to specifically trigger redistribution but some authors may forget to make this extra function call because it's not required in normal cases.

 <https://gist.github.com/rniwa/2f14588926e1a11c65d3#pros-1>Pros

Components don't have to implement complicated redistribution algorithms themselves.
Allows distributed nodes to be re-ordered (select doesn't allow this).
 <https://gist.github.com/rniwa/2f14588926e1a11c65d3#cons-1>Cons

Redistribution algorithm is not simple
At a slightly higher abstraction level


- R. Niwa