Re: Thoughts on ShadowDOM, Rushing Specs, and Widespread Adoption Before Specifications

On Feb 17, 2014 4:30 AM, "Calvin Spealman" <ironfroggy@gmail.com> wrote:
>
> This post is at the request of Brian Kardell, after some discussion last
night on Twitter that had difficulty fitting into the medium.

Calvin,

Thanks for moving the discussion to a more substantial medium.

I tried hard to respond inline or even by snipping and arranging comments
liberally, but in the end I felt like this was unproductive, so please
forgive my simple prose response out of context. I'll do my best to touch
on your points and share my own thoughts, but if you feel like I have
mistook or misrepresented something, feel free to point it out.

First off, we agree on the general idea that widespread experimentation
happens best outside of browser implementations and ultimately yields more
acceptable (even if sometimes potentially less academically good) results
for standards.  That's sort of the point of our group, in fact.

It seems that the trick is how to get there from here and effectively
better navigate the hurdles and that's what I wanted us to discuss...

You cite a whole lot of examples of things which, I think you are saying
provide a more exemplary model than Shadow DOM.  This is generally what I
was trying to debate on Twitter.  For example:  You mention HTML and CSS as
being based on previous works.  This is really mostly true at a conceptual
level.  CSS wasn't proposed through prollyfill with widespread
experimentation and agreement.  It was a comparatively small group of real
mavens who played with similar-ish ideas in special builds of a browser and
couldn't really do "real projects" until the thing existed natively.  No
one is really suggesting anything at this level today, so I think it's not
good to caught up in this particular example, let's move on to another...

You also cite the <video> element.  But again, it only standardized what
was happening previously in plugins at a very conceptual level.  There was
no API nor spec'ed details etc that picked up and moved over.  It didn't
lead with a forward-compatible prolyfill to show how it would work (APIs,
WEBIDL and so on) which could be safely experimented with by a large group
of folks and used in real projects,  it led with a native implementation.
 A polyfill came later, and by many accounts, not a great one really
because it lacks fidelity not possible without aspects of Web Components.
It *did*, however, come with an ability to fallback (this is actually just
HTML, not so much specific to video), but -- and here is an imporant point
--- that is definitely not experimentation. Combined with being a native
tag, that would have been the opposite of widespread experimentation really
- that would have been widespread use.  Far too late to employ substantial
feedback at that point.

To illustrate this point, let's look at the in-between then and now.  In
CSS and DOM, we had experimental *prefixes* in release builds.  Some
developers developed for a single browser and made real use of this
feature, but mostly it was a few mavens playing around with it.  A few
contributed comments back to a WG, many didn't.  Some folks blogged about
their experience.  One of two things inevitably follows, additional
implementation and standards work occurs or the idea is dropped.  As
additional work begins, we find that there are kinks to be worked out.  The
more implementations ship, the more people consider this experiment not so
experimental and begin shipping, as you suggest, real choose.  The amount
of feedback a group receives is still comparatively low, but there is a net
increase and so we find more things require tweaking.  But now we risk
breaking all those things and standing on the "you really should have
realized that that was experimental" ground which is self-defeating.  So
the practical upshot is that once it is in a release build, something is
either usable natively, or it isn't.  It's too late at that stage of the
game for feedback too.

So prollyfills, especially if we go through the effort to make them forward
compatible, can give you much of what you're asking for in terms of the
feedback loop and widespread experimentation.  But, to write them requires
primitives with which to build them.  At some point you bottom out in your
ability to do so:  There are existing standards that it is not even
possible to *polyfill* in some browsers.  Obviously, as time marches
forward, it becomes more an more practical as the Platform adds new
features we can use to make other features.  In other words, there is
really no way around the fact that there is a minimum criteria and
sometimes a browser or the platform itself lacks the necessary primitives
to fill it.

Let's go back to the <video> tag example... Without a plugin, our ability
to prollyfill (or even polyfill) a new video feature would have been pretty
much 0 -- it requires low level APIs that were fundamentally lacking in the
platform.  Our ability to polyfill using Flash and have *really wide spread
confidence you would want in production for real product* had a lot to do
with the near ubiquity of the Flash player created by a confluence of
history, pressures and business agreements that is unlikely to recur again.
 So what do we do in these cases and what are the aims?  We *can* base a
prollyfill off a plugin, but that won't likely yield a lot of the sorts of
widespread *real* use you desire.  It's better than nothing, and I'm all
for it if it helps us work it out, get people involved and so on.

That's great, in fact.  Chris Wilson did this with some of the Audio/MIDI
stuff and you could actually use it for *some* real stuff if you were
careful and didn't mind requiring a plugin.

But this won't work for all cases - at some point we reach the bottom and
have to say "new low-level APIs *required*".  This is why probably the most
cited thing about the Extensible Web Manifesto is to identify and
prioritize missing fundamental primitives in the platform.

This is where Shadow DOM is an interesting case, it's not *necessarily* a
primitive.  It's low, it's fundamentally useful, but one could make the
case that it's not - as currently spec'ed - low enough still.  Whether you
think it is one or not, the point is, there are necessary ones down there
and my question to you I guess is - other than putting things behind a flag
until we can get a stable enough agreement to release something natively
upon which to build all of the other things --- what *could* be done to
improve this?

-Brian