Thoughts on ShadowDOM, Rushing Specs, and Widespread Adoption Before Specifications

This post is at the request of Brian Kardell, after some discussion last
night on Twitter that had difficulty fitting into the medium.

The position I tried to get across: ShadowDOM differs from a lot of other
W3C specifications in the timeline of being "frozen" in a specification and
subsequent native implementation and the chance for widespread
experimentation with a new API that web developers have to work out
problems, discover best practices, and give their feedback into the process.

My supporting claims:
- Feature flags are not sufficient for "widespread experimentation"
- Widely available (in terms of browser support) polyfill options are
essential

In reverse order to explain my claims, starting with the issue of
Polyfills. The only polyfill for ShadowDOM has been the frankly
disappointing option provided as part of the Polymer project. Primarily
this polyfill only seems to polyfill the official API as a wrapper around
the webkit prefixed incarnation, and outside of any browser that *actually* has
ShadowDOM support of some sort, the polyfill is (nearly?) useless. That
Polymer itself is advertised as "pre-alpha", refuses to support older but
widely available browsers (like the pre-Android 4.4 browser), and is even
claimed by themselves to *not* be ready for use demonstrates that this API
is still far from ready for anything resembling "widespread
experimentation".

Which brings me to the first supporting claim that we do not have the
option for widespread experimentation even when we are already seeing
native implementations. A primary issue is this is often behind a feature
flag in the browser, not enabled by default. The need and cause for feature
flags is obvious, but it limits a feature to being subject to experiments
firstly only built by a very small subset of developers and secondly only
*consumed* but a likewise small subset of other developers.

Widespread experimentation requires anything close to "real world"
deployment. We need the ability to try ShadowDOM (and all new APIs) in
environments where they exist as part of a larger project subject to the
pressures and requirements of deployment, users, and ongoing maintenance.
Small, toy experiments built solely for the curiosity of the developer who
has both the know how and (primarily) the free time outside their job are
whole inadequate

The last point depends on two important problems with the inadequate nature
of getting our feet wet with these new APIs: that we can only really do so
on free, unpaid time unduly empowers a smaller privileged subset to drive
the direction of all and that even these are doing so in an environment
that is entirely unable to replicate the use of the new technology in the
real world. The result is that we are thrust upon us new APIs and new
abilities in our browsers and we are, often, irrevocably locked into these
before the vast majority of us had even an inkling of the progress towards
their standardization and implementation.

This is not an inherent aspect of extending our web development grammar
with new APIs. We have added a lot over the years but in nearly all cases
we had some ability to experiment widely before anything ever approached a
standards specification. CSS was based on other content technologies before
it, just as HTML itself was based on previous mark up products. The video
tag sought to standardize that which we already had been doing for over a
decade with plug-ins, which continued to provide a path to experiment with
the new tag before our browsers ever supported them officially. We were
able to work out the issues and mistakes before we were stuck with them. We
have the ability to polyfill other parts of the Web Component APIs, such as
Custom Elements and HTML Imports in browsers that were never intended to
support them. And, in doing so, we're finding problems and are able to
resolve them and change the specifications before the official
implementations land.

ShadowDOM is different. ShadowDOM is (almost?) exclusively delivered as a
pre-packaged native implementation based on theories about how to solve a
problem with little real practice in understanding the real benefits,
costs, or ever having a chance at competing against alternatives.